Information Intermediaries


Published on

Information intermediaries for government linked data

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Information Intermediaries

  1. 1. Information intermediaries for government linked data<br />Dave Reynolds, Epimorphics Ltd<br />
  2. 2. Governments around the world are releasing data<br />
  3. 3. Why?<br />transparency, openness, it’s public data<br />tap creativity, enthusiasm of web developersstimulate applications for citizens & commerce<br />track crime in your area<br />understand where funding is going<br />plan travel <br />choose a school<br />
  4. 4. Theme for this talk<br />how to accelerate this uptake?reduce cost of exploiting public data?stimulate an ecosystem of value added services?<br />data dump and information intermediaries<br />linked data approach<br />intermediaries for a linked data world<br />
  5. 5. Traditional publication approach:data dumps<br />publish individual datasets – typically CSV<br />easy for publisher<br />consumer has complete control<br />no complex formats or query languages<br />manage data as they want to<br />familiar technology stack <br />growing set of intermediaries web services to help you work with datasets<br />not specific to public sector data<br />
  6. 6. Intermediary services<br />
  7. 7. Intermediary services<br />
  8. 8. Intermediary services<br />
  9. 9. Limitations to data dumps<br />Silo design pattern<br />each application does its own data integration <br />hard to share or reuse efforts between applications<br />Static<br />local stores which require management and update<br />*<br />
  10. 10. Linked data : public sector data web<br />How:<br />URIs to identify things described<br />dereference to RDF (& other formats)<br />SPARQL endpoints for query<br />vocabularies and patterns for <br />statistics, versioning, provenance ...<br />standard URI sets <br />time periods, regions, departments, schools ...<br />
  11. 11. Public sector data web<br />DCSF<br />AdminGeography<br />Edubase<br />Schools<br />TimePeriods<br />Ofsted<br />Gov.Bodies<br />
  12. 12. Benefits of linked data approach<br />integrated (linked!) data<br />standard identifiers enables linking other sets<br />seed connections between third party sets<br />fine grain addressing of data<br />annotations (e.g. provenance)<br />fine grained programmatic access<br />consume live or cache, not forced to use static <br />data model directly linked from data<br />
  13. 13. But ...<br />barrier to entry too high - “just give us CSV”<br />alien data model<br />alien query methods<br />alien representation formats<br />overall mismatch to typical web developer tool kit<br />
  14. 14. Solution<br />middleware to provide web-friendly access<br />run at publisher end or as an intermediary<br />publish as linked data -> automatic API<br />configure automatically from ontology<br />customize configuration (e.g.URI patterns)if needed<br />
  15. 15. Linked data API<br />Access<br />RESTful API design<br />serve lists of resources or individual resources<br />automatic sorting, paging of lists<br />simple web API to control filtering, viewing<br />Formatting<br />developer-friendly JSON & XML<br />retain resource-centric model<br />remove round-tripping requirements<br />rooted graph<br />
  16. 16. Structure<br />request<br />SELECT ?item<br />WHERE { ... }<br />GET /doc/schools/district/Oxford.json ? min-capacity=1200<br />Data source<br />SPARQL<br />endpoint<br />selector<br />viewer<br />DESCRIBE <x> <y><br />formatter<br />cache<br />response<br />Endpoint<br />API specification<br />vocabulary of data set<br />
  17. 17. Operation<br />/doc/schools/district/Oxford.json ? min-capacity=1200<br />Matchendpoint<br />/doc/schools/district/{d}<br />Retrievematches<br />SELECT ?r WHERE {<br /> ?r a school:School;<br />school:district [rdfs:label ‘Oxford’];<br />school:capacity ?c .<br /> FILTER (?c >= 1200)<br />} OFFSET 0 LIMIT 10<br />buildresponse<br />metadata: query and configuration<br />List<br />page N-1<br />page N<br />page N+1<br />select format:<br />JSON<br />school i<br />school i<br />school i<br />
  18. 18. JSON serialization<br /> "results":[<br /> {<br /> "_about":"http://.../district/Oxford?min-schoolCapacity=1200&_page=0",<br /> "first":"http://.../district/Oxford?&min-schoolCapacity=1200&_page=0",<br /> "isPartOf":"http://.../district/Oxford?&min-schoolCapacity=1200",<br /> "page":0, "pageSize":10,<br /> "type":"",<br /> "contains":[<br /> {<br /> "_about":"",<br /> "label":"Peers School",<br /> "districtAdministrative":{<br /> "_about":"",<br /> "label":"Oxford“ },<br /> "phaseOfEducation":{<br /> "_about":"",<br /> "label":"Secondary” },<br /> "schoolCapacity":1220,<br /> "type":[<br /> {<br /> "_about":"",<br /> "label":"School” },<br /> }, ...<br />
  19. 19. Linked data API : outcomes<br />lowers barrier to entry<br />very positive reception<br />build linked data applications with e.g. jQuery<br />no need to for full RDF stack<br />stepping stone to linked data world<br />retain concept of resources with URIs<br />retain schema-less model<br />look at the SPARQL you made, look at API config<br />open specification (Epimorphics, Talis, TSO)<br />multiple implementations, including open source<br /><br />
  20. 20. What other mediators are needed for a linked data world?<br />
  21. 21. Conclusions <br />intermediary services, such as LD access API, can make the power and flexibility of linked data available to broader range of developers<br />meet public sector goals of stimulating network of value added applications for citizens and business<br />lots more to do ...<br />