Data All the Way DownJeni Tennison@JeniThttp://www.jenitennison.com/blog/
Data All the Way Down• challenges of complex open data• layered approach to data publishing• essential steps• benefits
Complex Datasets• too much for a single spreadsheet• need to navigate • browse through data • look at slices of larger dat...
User Challenge• complex data sets have range of users • different hardware / platforms • different tasks / goals • differe...
visualisation / data gap   end user vs reuser
Visualisations• approachable for real people• necessary for stakeholder buy-in• beauty is in whats left out • advertisemen...
Raw Data• importable into own data store • often only interested in particular slice • data set may be massive / changing•...
bridging the gap                         layered data accessPhoto by Nikita Kravchuk http://www.flickr.com/photos/mi55er/38...
Layered Architecture• user interface • navigation and global understanding• API • curated, targeted, programmable access• ...
legislation.gov.uk   lists as Atom feeds
legislation.gov.uk   content as XML
legislation.gov.uk   layer other views
organograms   navigable visualisation
organograms   JSON data
organograms   RDF / XML / HTML
organograms   SPARQL query
organograms   raw data
Key Techniques• resource-driven design (good URIs)• every page built based on API calls• explicit links to API access • fo...
Benefits• fork at any point • dont like the visualisation / API? create your own!• everyone is human • reusers gain underst...
Questions?
Upcoming SlideShare
Loading in...5
×

Data All the Way Down

3,192

Published on

Presentation at OKCon 2011 on how to build web applications that provide complex data using a layered architecture.

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,192
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
27
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Data All the Way Down

  1. 1. Data All the Way DownJeni Tennison@JeniThttp://www.jenitennison.com/blog/
  2. 2. Data All the Way Down• challenges of complex open data• layered approach to data publishing• essential steps• benefits
  3. 3. Complex Datasets• too much for a single spreadsheet• need to navigate • browse through data • look at slices of larger dataset • view summary statistics• need to explain • definitions of terms, provisos & disclaimers
  4. 4. User Challenge• complex data sets have range of users • different hardware / platforms • different tasks / goals • different ability / understanding• no one interface satisfies everyone• data owners cannot satisfy everyone• create ecosystem around open data
  5. 5. visualisation / data gap end user vs reuser
  6. 6. Visualisations• approachable for real people• necessary for stakeholder buy-in• beauty is in whats left out • advertisement or taster of rich datasets • often not possible in official data• leaves questions unanswered • what if we looked at the data in a different way?
  7. 7. Raw Data• importable into own data store • often only interested in particular slice • data set may be massive / changing• run whatever analysis you want • requires at least some programming skills • analysis might not be appropriate for the data• documentation probably lacking
  8. 8. bridging the gap layered data accessPhoto by Nikita Kravchuk http://www.flickr.com/photos/mi55er/3845619153/
  9. 9. Layered Architecture• user interface • navigation and global understanding• API • curated, targeted, programmable access• query • free-form programmable access• raw data
  10. 10. legislation.gov.uk lists as Atom feeds
  11. 11. legislation.gov.uk content as XML
  12. 12. legislation.gov.uk layer other views
  13. 13. organograms navigable visualisation
  14. 14. organograms JSON data
  15. 15. organograms RDF / XML / HTML
  16. 16. organograms SPARQL query
  17. 17. organograms raw data
  18. 18. Key Techniques• resource-driven design (good URIs)• every page built based on API calls• explicit links to API access • for bonus points, link to your transformation code• consistent terminology • clear mapping from UI to API• caching & access control at each level
  19. 19. Benefits• fork at any point • dont like the visualisation / API? create your own!• everyone is human • reusers gain understanding from user interface• visualisation benefits the stack • API oriented towards achieving a goal • visual validation of data improves quality
  20. 20. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×