Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Linked Data Hypercubes


Published on

Presentation on the Data Cube vocabulary to support Linked Data publication of statistics and measurement data sets. Given at SemTech 2011, San Francisco.

Published in: Technology, Education

Linked Data Hypercubes

  1. 1. Linked data hypercubes<br />Dave Reynolds, Epimorphics Ltd<br />
  2. 2.
  3. 3. Linked Data - great for describing “things”<br />data<br />Government<br />departments<br />Posts and reporting structure<br />e.g.<br />Ministers and senior civil servants<br />
  4. 4. Linked Data - great for describing “things”<br />data<br />model<br />choose entities to model<br />department, post, post-holder ...<br />Ontologies<br />org:, foaf: ...<br />domain specific – gov:<br />URI scheme<br />reference data to link to<br />
  5. 5. Linked Data - great for describing “things”<br />data<br />model<br />publish<br />convert to RDF in a triple store<br />entity URIs as linked data<br />SPARQL endpoint<br />Linked data API<br />
  6. 6. Linked Data - great for describing “things”<br />data<br />model<br />publish<br />use<br />
  7. 7. But what about ... data<br />Government budget analysis<br />local authority spend with suppliers<br />regional demographic trends<br />performance metrics<br />air quality measurements<br />energy consumption<br />
  8. 8. Publishing tabular data as linked data<br />?<br />why?<br />how?<br />does it work?<br />
  9. 9. Benefits<br />data slices and values becomes addressable<br />annotate, explain, qualify values<br />provenance for values<br />trace back for derived reports<br />integrate, compare, slice across datasets<br />common terms for dimensions and units<br />common identifiers for values (regions, departments ...)<br />link to non-tabular data<br />put the data in context<br />
  10. 10. Data cube vocabulary <br />collaborative development sponsored by<br />simple, flexible vocabulary<br />mirrors core information models from:<br />SDMX (Statistical Data and Metadata eXchange)<br />DDI (Data Documentation Initiative)<br />extension to SCOVO vocabulary<br />
  11. 11. Data cube model<br />A set of observations<br />indexed by dimensions<br />describing measures<br />interpreted according to attributes<br />attributes<br />measure(s)<br />dimension(e.g. region)<br /><ul><li> population = 32,567</li></ul>unit of measure = count<br />status = preliminary<br />...<br />dimension(e.g. time)<br />
  12. 12. Data cube vocabulary1. Top level<br />DataSet<br />provenance and metadata<br />structure <br />qb:component<br />qb:DataStructureDefinition<br />qb:sliceKey<br />qb:structure<br />qb:SliceKey<br />qb:DataSet<br />qb:slice<br />qb:sliceStructure<br />qb:dataset<br />qb:Slice<br />qb:subSlice<br />qb:observation<br />qb:Observation<br />dimension valuesmeasure value(s)<br />attribute values<br />
  13. 13. Data cube vocabulary1. Top level<br />DataSet<br />provenance and metadata<br />structure <br />Observation<br />measured values, at dimensions with attributes<br />direct link to DataSet<br />qb:component<br />qb:DataStructureDefinition<br />qb:sliceKey<br />qb:structure<br />qb:SliceKey<br />qb:DataSet<br />qb:slice<br />qb:sliceStructure<br />qb:dataset<br />qb:Slice<br />qb:subSlice<br />qb:observation<br />qb:Observation<br />dimension valuesmeasure value(s)<br />attribute values<br />
  14. 14. Data cube vocabulary1. Top level<br />DataSet<br />provenance and metadata<br />structure <br />Observation<br />measured values, at dimensions with attributes<br />direct link to DataSet<br />Slice<br />optional grouping by fixing dimensions<br />guide to presentation<br />allows for abbreviated data<br />qb:component<br />qb:DataStructureDefinition<br />qb:sliceKey<br />qb:structure<br />qb:SliceKey<br />qb:DataSet<br />qb:slice<br />qb:sliceStructure<br />qb:dataset<br />qb:Slice<br />qb:subSlice<br />qb:observation<br />qb:Observation<br />dimension valuesmeasure value(s)<br />attribute values<br />
  15. 15. Data cube vocabulary2. Data Structure Definition<br />explicit definition of cube structure, inline in the data<br />enables<br />validation<br />visualization<br />discovery<br />abbreviation<br />still open world<br />qb:DataSet<br />qb:structure<br />qb:DataStructureDefinition<br />qb:component<br />qb:ComponentSpecification<br />qb:componentRequired qb:componentAttachment qb:order <br />qb:dimension<br />qb:measure<br />qb:attribute<br />
  16. 16. Data cube vocabulary3. Coding values<br />numeric or symbolic<br />explicit link to coding scheme<br />allows for hierarchical codes<br />SDMX coding schemes and role markers available<br />qb:ComponentProperty<br />qb:concept<br />qb:DimensionProperty<br />qb:measureType<br />skos:Concept<br />qb:AttributeProperty<br />sdmx:Concept<br />qb:MeasureProperty<br />sdmx:ConceptRole<br />qb:CodedProperty<br />qb:codeList<br />sdmx:FrequencyRolesdmx:CountRolesdmx:EntityRolesdmx:TimeRole<br />sdmx:MeasureTypeRole<br />sdmx:NonObsTimeRole<br />sdmx:IdentityRole<br />sdmx:PrimaryMeasureRole<br />skos:ConceptScheme<br />sdmx:CodeList<br />
  17. 17. Example<br />eg:dsd-le a qb:DataStructureDefinition;<br /> # The dimensions<br /> qb:component [qb:dimension eg:refArea; qb:order 1];<br /> qb:component [qb:dimension eg:refPeriod; qb:order 2];<br /> qb:component [qb:dimension sdmx-dimension:sex; qb:order 3];<br /> # The measure(s)<br /> qb:component [qb:measure eg:lifeExpectancy];<br /> # The attributes<br /> qb:component [qb:attribute sdmx-attribute:unitMeasure;<br /> qb:componentAttachment qb:DataSet;] .<br />eg:dataset-le1 a qb:DataSet;<br /> rdfs:label "Life expectancy"@en;<br /> rdfs:comment "Life expectancy in Welsh Unitary authorities"@en;<br /> qb:structure eg:dsd-le ;<br /> sdmx-attribute:unitMeasure <> .<br /> eg:o1 a qb:Observation;<br /> qb:dataset eg:dataset-le1 ;<br /> eg:refArea admingeo:newport_00pr ; <br /> eg:refPeriod <> ;<br /> sdmx-dimension:sex sdmx-code:sex-M ;<br /> eg:lifeExpectancy 76.7 .<br />
  18. 18. Case study: Local government payments<br />data<br />UK local authorities publish data on all spending above £500<br />linked data version to enable comparison<br />
  19. 19. Case study: Local government payments<br />data<br />model<br />cube structure<br />measure<br />amount net of recoverable VAT<br />attributes<br />currency<br />dimensions<br />time<br />payer<br />payee<br />expenditure code<br />item<br />package as an ontology<br />
  20. 20. Case study: Local government payments<br />data<br />model<br />publish<br />visualizations<br />LD API<br />API structure mirrors cube dimensional structure <br />
  21. 21. Case study: Local government payments<br />data<br />model<br />publish<br />use<br />
  22. 22. Case study: Environmental monitoring<br />data<br /><ul><li>Environment Agency bathing water quality monitoring
  23. 23. samples
  24. 24. assay
  25. 25. compliance assessment</li></li></ul><li>Case study: Environmental monitoring<br />data<br />model<br />measures<br />total coliform count, entero virus count, ...<br />sample classification<br />dimensions<br />sampling point<br />sampling week<br />sampling year<br />attributes<br />abnormal weather<br />
  26. 26. Case study: Environmental monitoring<br />data<br />model<br />publish<br />visualizations<br />LD API<br />API structure mirrors cube dimensional structure <br />
  27. 27. Case study: Environmental monitoring<br />data<br />model<br />publish<br />use<br />
  28. 28. Data Cube : Summary<br />foundational approach to publishing multi-dimensional data as linked data<br />enables<br />addressing – annotate, explain, provenance, context<br />integration – slice, dice and compare across sets<br />puts data in context<br />explicit declarative structure =><br />validation<br />discovery<br />automation - web APIs, visualizations, exploration tools<br />
  29. 29. Acknowledgements<br />John Sheridan (The National Archive) <br />for sponsoring the development of data cube<br />Richard Cyganiak, JeniTennison<br />co-developers of the data cube vocabulary<br />JeniTennison, Dan Smith<br />developers of the organogram example<br />Stuart Williams, Ian Dickinson<br />developers of the bathing water use case<br />Photos:<br />lesteph @ flickrdullhunk @ flickr Martin Pettitt @ flickr<br />kikasso @ flickrTax_Rebate @ flikr<br />