Data All the Way Down

Data All the Way DownJeni Tennison@JeniThttp://www.jenitennison.com/blog/

Data All the Way Down• challenges of complex open data• layered approach to data publishing• essential steps• beneﬁts

Complex Datasets• too much for a single spreadsheet• need to navigate • browse through data • look at slices of larger dataset • view summary statistics• need to explain • deﬁnitions of terms, provisos & disclaimers

User Challenge• complex data sets have range of users • different hardware / platforms • different tasks / goals • different ability / understanding• no one interface satisﬁes everyone• data owners cannot satisfy everyone• create ecosystem around open data

visualisation / data gap end user vs reuser

Visualisations• approachable for real people• necessary for stakeholder buy-in• beauty is in whats left out • advertisement or taster of rich datasets • often not possible in ofﬁcial data• leaves questions unanswered • what if we looked at the data in a different way?

Raw Data• importable into own data store • often only interested in particular slice • data set may be massive / changing• run whatever analysis you want • requires at least some programming skills • analysis might not be appropriate for the data• documentation probably lacking

bridging the gap layered data accessPhoto by Nikita Kravchuk http://www.ﬂickr.com/photos/mi55er/3845619153/

Layered Architecture• user interface • navigation and global understanding• API • curated, targeted, programmable access• query • free-form programmable access• raw data

legislation.gov.uk lists as Atom feeds

legislation.gov.uk content as XML

legislation.gov.uk layer other views

organograms navigable visualisation

organograms JSON data

organograms RDF / XML / HTML

organograms SPARQL query

organograms raw data

Key Techniques• resource-driven design (good URIs)• every page built based on API calls• explicit links to API access • for bonus points, link to your transformation code• consistent terminology • clear mapping from UI to API• caching & access control at each level

Beneﬁts• fork at any point • dont like the visualisation / API? create your own!• everyone is human • reusers gain understanding from user interface• visualisation beneﬁts the stack • API oriented towards achieving a goal • visual validation of data improves quality

Questions?

http://storify.com	30
http://paper.li	4
http://www.slideshare.net	2
http://a0.twimg.com	1
http://www.linkedin.com	1

Data All the Way Down

by Jeni Tennison on Jul 02, 2011

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 38

Statistics

Data All the Way Down Presentation Transcript