Linked Data Hypercubes - Semtech London

1,412 views

Published on

Presentation on the Data Cube vocabulary, and its uses, given at the Semantic Technologies Business Conference in London.

Published in: Technology, Education

Linked Data Hypercubes - Semtech London

  1. 1. Linked data hypercubes<br />Dave Reynolds, Epimorphics Ltd<br />
  2. 2.
  3. 3. Linked Data - great for describing “things”<br />data<br />e.g. Schools in England and Wales<br />
  4. 4. Linked Data - great for describing “things”<br />data<br />model<br />ontology development<br />classifications<br />phase of education<br />location, contact<br />reporting<br />class sizes etc<br />URI scheme<br />reference data to link to<br />admin geography, LLSC, charity ...<br />
  5. 5. Linked Data - great for describing “things”<br />data<br />model<br />publish<br />convert to RDF in a triple store<br />entity URIs as linked data<br />SPARQL endpoint<br />Linked data API<br />
  6. 6. Linked Data - great for describing “things”<br />data<br />model<br />publish<br />use<br />
  7. 7. But what about ... data<br />Government budget analysis<br />local authority spend with suppliers<br />regional demographic trends<br />performance metrics<br />air quality measurements<br />energy consumption<br />
  8. 8. Publishing tabular data as linked data<br />?<br />why?<br />how?<br />does it work?<br />
  9. 9. Benefits<br />data slices and values becomes addressable<br />annotate, explain, qualify values<br />provenance for values<br />trace back for derived reports<br />integrate, compare, slice across datasets<br />common terms for dimensions and units<br />common identifiers for values (regions, departments ...)<br />link to non-tabular data<br />put the data in context<br />
  10. 10. Data cube vocabulary <br />collaborative development sponsored by data.gov.uk<br />simple, flexible vocabulary<br />mirrors core information models from:<br />SDMX (Statistical Data and Metadata eXchange)<br />DDI (Data Documentation Initiative)<br />extension to SCOVO vocabulary<br />
  11. 11. Data cube model<br />A set of observations<br />indexed by dimensions<br />describing measures<br />interpreted according to attributes<br />attributes<br />measure(s)<br />dimension(e.g. region)<br /><ul><li> population = 32,567</li></ul>unit of measure = count<br />status = preliminary<br />...<br />dimension(e.g. time)<br />
  12. 12. Data cube vocabulary1. Top level<br />DataSet<br />provenance and metadata<br />structure <br />qb:component<br />qb:DataStructureDefinition<br />qb:sliceKey<br />qb:structure<br />qb:SliceKey<br />qb:DataSet<br />qb:slice<br />qb:sliceStructure<br />qb:dataset<br />qb:Slice<br />qb:subSlice<br />qb:observation<br />qb:Observation<br />dimension valuesmeasure value(s)<br />attribute values<br />
  13. 13. Data cube vocabulary1. Top level<br />DataSet<br />provenance and metadata<br />structure <br />Observation<br />measured values, at dimensions with attributes<br />direct link to DataSet<br />qb:component<br />qb:DataStructureDefinition<br />qb:sliceKey<br />qb:structure<br />qb:SliceKey<br />qb:DataSet<br />qb:slice<br />qb:sliceStructure<br />qb:dataset<br />qb:Slice<br />qb:subSlice<br />qb:observation<br />qb:Observation<br />dimension valuesmeasure value(s)<br />attribute values<br />
  14. 14. Data cube vocabulary1. Top level<br />DataSet<br />provenance and metadata<br />structure <br />Observation<br />measured values, at dimensions with attributes<br />direct link to DataSet<br />Slice<br />optional grouping by fixing dimensions<br />guide to presentation<br />allows for abbreviated data<br />qb:component<br />qb:DataStructureDefinition<br />qb:sliceKey<br />qb:structure<br />qb:SliceKey<br />qb:DataSet<br />qb:slice<br />qb:sliceStructure<br />qb:dataset<br />qb:Slice<br />qb:subSlice<br />qb:observation<br />qb:Observation<br />dimension valuesmeasure value(s)<br />attribute values<br />
  15. 15. Data cube vocabulary2. Data Structure Definition<br />explicit definition of cube structure, inline in the data<br />enables<br />validation<br />visualization<br />discovery<br />abbreviation<br />still open world<br />qb:DataSet<br />qb:structure<br />qb:DataStructureDefinition<br />qb:component<br />qb:ComponentSpecification<br />qb:componentRequired qb:componentAttachment qb:order <br />qb:dimension<br />qb:measure<br />qb:attribute<br />
  16. 16. Data cube vocabulary3. Coding values<br />numeric or symbolic<br />explicit link to coding scheme<br />allows for hierarchical codes<br />SDMX coding schemes and role markers available<br />qb:ComponentProperty<br />qb:concept<br />qb:DimensionProperty<br />qb:measureType<br />skos:Concept<br />qb:AttributeProperty<br />sdmx:Concept<br />qb:MeasureProperty<br />sdmx:ConceptRole<br />qb:CodedProperty<br />qb:codeList<br />sdmx:FrequencyRolesdmx:CountRolesdmx:EntityRolesdmx:TimeRole<br />sdmx:MeasureTypeRole<br />sdmx:NonObsTimeRole<br />sdmx:IdentityRole<br />sdmx:PrimaryMeasureRole<br />skos:ConceptScheme<br />sdmx:CodeList<br />
  17. 17. Example<br />eg:dsd-le a qb:DataStructureDefinition;<br /> # The dimensions<br /> qb:component [qb:dimension eg:refArea; qb:order 1];<br /> qb:component [qb:dimension eg:refPeriod; qb:order 2];<br /> qb:component [qb:dimension sdmx-dimension:sex; qb:order 3];<br /> # The measure(s)<br /> qb:component [qb:measure eg:lifeExpectancy];<br /> # The attributes<br /> qb:component [qb:attribute sdmx-attribute:unitMeasure;<br /> qb:componentAttachment qb:DataSet;] .<br />eg:dataset-le1 a qb:DataSet;<br /> rdfs:label "Life expectancy"@en;<br /> rdfs:comment "Life expectancy in Welsh Unitary authorities"@en;<br /> qb:structure eg:dsd-le ;<br /> sdmx-attribute:unitMeasure <http://dbpedia.org/resource/Year> .<br /> eg:o1 a qb:Observation;<br /> qb:dataset eg:dataset-le1 ;<br /> eg:refArea admingeo:newport_00pr ; <br /> eg:refPeriod <http://reference.data.gov.uk/id/year/2004> ;<br /> sdmx-dimension:sex sdmx-code:sex-M ;<br /> eg:lifeExpectancy 76.7 .<br />
  18. 18. Case study: Local government payments<br />data<br />UK local authorities publish data on all spending above £500<br />linked data version to enable comparison<br />
  19. 19. Case study: Local government payments<br />data<br />model<br />cube structure<br />measure<br />amount net of recoverable VAT<br />attributes<br />currency<br />dimensions<br />time<br />payer<br />payee<br />expenditure code<br />item<br />package as an ontology<br />
  20. 20. Case study: Local government payments<br />data<br />model<br />publish<br />visualizations<br />LD API<br />API structure mirrors cube dimensional structure <br />
  21. 21. Case study: Local government payments<br />data<br />model<br />publish<br />use<br />
  22. 22. Case study: Environmental monitoring<br />data<br /><ul><li>Environment Agency bathing water quality monitoring
  23. 23. samples
  24. 24. assay
  25. 25. compliance assessment</li></li></ul><li>Case study: Environmental monitoring<br />data<br />model<br />measures<br />total coliform count, entero virus count, ...<br />sample classification<br />dimensions<br />sampling point<br />sampling week<br />sampling year<br />attributes<br />abnormal weather<br />
  26. 26. Case study: Environmental monitoring<br />data<br />model<br />publish<br />visualizations<br />LD API<br />API structure mirrors cube dimensional structure <br />
  27. 27. Case study: Environmental monitoring<br />data<br />model<br />publish<br />use<br />
  28. 28. Data Cube : Summary<br />foundational approach to publishing multi-dimensional data as linked data<br />enables<br />addressing – annotate, explain, provenance, context<br />integration – slice, dice and compare across sets<br />puts data in context<br />explicit declarative structure =><br />validation<br />discovery<br />automation - web APIs, visualizations, exploration tools<br />
  29. 29. Acknowledgements<br />John Sheridan (The National Archive) <br />for sponsoring the development of data cube<br />Richard Cyganiak, JeniTennison<br />co-developers of the data cube vocabulary<br />Paul Davidson <br />instigator of the Payments ontology<br />Stuart Williams, Ian Dickinson<br />developers of the bathing water use case<br />Photos:<br />dullhunk @ flickr Martin Pettitt @ flickr<br />kikasso @ flickrTax_Rebate @ fliCkr<br />

×