3. Where can I get the data?
Answer:
From a SPARQL endpoint (portal to get linked
data)
Massive bonus:
A Linked Data API
4. What data do I want for my
visualisation?
• Departments
http://reference.data.gov.uk/doc/department
• Units within a department
http://reference.data.gov.uk/doc/department/dfe/unit
• Posts within a unit
http://reference.data.gov.uk/doc/department/dfe/unit/educati
on-standards-directorate/post
5. I want departments + their units + the unit posts
+ the posts that report to those posts!
http://reference.data.gov.uk/doc/department.json?
• _view=
minimal
• _properties=
unit.label
unit.post.label
unit.post.reportsTo
• _pageSize=
100
• http://reference.data.gov.uk/doc/department.json?
_view=minimal&_properties=unit.label,unit.post.label,unit.post.reportsTo&_pageSize=10
- The Telegraph recently created a collage of all of the Whitehall department organograms, naming it “Cameron’s Biggest Nightmare”
- And you saw from earlier that this data is becoming available as linked data In the reference.data.gov.uk database (List of organograms from the Cabinet Office: http://www.cabinetoffice.gov.uk/newsroom/news_releases/2010/101015-structure-charts.aspx)
In the last few months, I’ve been trying to create an “organogram explorer” that people can use to browse around departments - seeing who reports to who as well as the general overview of a department and it’s units
As the data was to be pulled in real-time from the reference.data.gov.uk database - whenever the data was updated, my visualisations would update automatically too.
The HTML Viewer and my visualisations are both direct representations of the data – what you see is what you can get
More than often, following a hyper-link in search of linked data will take you to a blank text box asking for a SPARQL query.
Sure a SPARQL query can be entered by hand, but these endpoints (serving machine-readable data) are mainly for applications or API’s to latch on to
Chances are, the first time round, you have no idea what data you want - you just want to take a peek at what’s on offer
With the introduction of the API and it’s HTML Viewer – useful links can be directly provided for lists of data, which can now be found on this page for example: http://data.gov.uk/linked-data
- So with a bit of previous linked data experience, I started tailoring my data needs – thinking about what I needed in terms of “things” (which is handy when using linked data)
- In my case, I was told specifically what data I could use and the types of information contained in that data, so I didn’t need to carry out much detective work myself
If this wasn’t the case, the Linked Data API’s HTML Viewer does a brilliant job at letting you explore and discover data for yourself
As mentioned, for any API calls, several formats can be requested by simply adding their extension to the end of the URL (.rdf, .ttl, .xml, .json)
- The API’s flexibility is outstanding – you can get your hands on almost anything in any shape
- For the initial API call for the government structure visualisation, I needed a list of departments with a path of connecting information
For each department, the path was “department > units > posts > reporting posts”
Which can be accessed by specifying _properties=unit,unit.post,unit.post.reportsTo
“.label” means I want the human-friendly, readable version of that item
- On the left is a list of departments in JSON format returned by the API (http://reference.data.gov.uk/doc/department.json)
The top part of the response is useful metadata about the data such as the number of items per page and what formats are available
Typically a result will contain a list of “items” (for List Endpoints) – and in this case the items are “departments”
Each “department” item returned by the API contains data such as the department’s name and the units within that department
- You saw there that the API returned a “nested” structure (lists of units within a list of departments), and uses the API’s vocabulary to name the JSON items and their properties
However, the visualisation plugin I wanted to use required a different JSON data structure (on the right)
I needed to convert the API’s JSON structure to a different nested structure, but using different property names such as “id,name,data and children”
Demo
/gov-structure
Aiming to be a “journey-starter” for the organogram visualisation, this is a “treemap” – allowing you to drill down into departments and units.
Clicking on a post will load it’s organogram
You can jump back to the /gov-structure visualisation by clicking on the breadcrumbs within the organogram visualisation
Information about the data being used in the visualisation can be found by clicking on the data source items in the bottom right of the screen
Note: this isn’t the whole government visualised, but it’s what is available through reference.data.gov.uk right now. As data is uploaded to the database, it will then appear within the visualisations
/organogram
A concious effort has been made to link to the data itself when possible – the post information panel contains links to the API’s HTML Viewer for the post and it’s unit
Again, the data sources are present in the bottom right –describing what the user can see infront of them, how it was retrieved and the different data formats available