Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Towards a frictionless data future

509 views

Published on

Presented at the Research Data Network workshop, St Andrews, 30 Nov 2016

Published in: Education
  • Be the first to comment

  • Be the first to like this

Towards a frictionless data future

  1. 1. Toward a Frictionless Data FuturePRESENTED BY Jo Barratt jo.barratt@okfn.org (@jobarratt/@okfnlabs) AT 3rd Research Data Network - St Andrews University - 30 November 2016 Licensed under cc-by v3.0 (any jurisdiction)
  2. 2. International non-profit founded in 2004 Who we are ● Vision o A world where open knowledge is ubiquitous, enabling citizens and organizations to create insights that drive change on global and local challenges, combat injustice and inequality and hold governments and corporations to account. ● Mission o Open up all essential, public interest information and see it used to create insight that drives positive change o Build communities, tools and skills to empower individuals and organizations to use open information to create insights that drive change.
  3. 3. Widely adopted - over 20 national governments and 60+ local governments & cities ckan.org/instances
  4. 4. £4m in 15mins
  5. 5. Frictionless Data is… ● Lightweight specifications for “packaging” datasets ● Integrations for loading datasets into tools and platforms relevant to researchers The Goals... ● Introduce a significant, measurable improvement in how research data is shared, consumed, and analyzed. ● Make it easier to maintain and improve data quality.
  6. 6. The Problem
  7. 7. Treemap of issues … Legal barriers (open data, sharing agreements etc) Data Quality Hard to find Interoperability No tool integration
  8. 8. Cargo loading ~1955 Manual, Slow, Costly (and Dangerous)
  9. 9. Data is Shipping Pre- Containerization
  10. 10. Containerization
  11. 11. Standards - Standards (a few, simple ones) - Tools (primarily for integration) - Documentation - Datasets Data Containerization http://www.flickr.com/photos/photohome_uk/1494590209/
  12. 12. Key Principles 1. Simplicity 2. Web Oriented 3. Existing Tools 4. Open 5 Distributed
  13. 13. Tabular Data Package Data Package JSON Table Schema CSV http://frictionlessdata.io/guides/tabular-data-package/ http://frictionlessdata.io/guides/data-package/ http://frictionlessdata.io/guides/json-table-schema/ Tabular Data Package
  14. 14. Tooling … Data Package Tabular Data Package JSON Table Schema CSV Tool e.g. import to R, SQL etc Tool e.g. data checking ala GoodTables
  15. 15. Validation http://goodtables.okfnlabs.org
  16. 16. Validation
  17. 17. Continuous Validation ● If you’re working in a group, you need continuous validation… for data! ● In < 1 hour, we integrated elements (datapackage.json + Python libraries + GoodTables API) to support continuous data validation
  18. 18. http://uk-25k.datadashboards.io/
  19. 19. Platform Integrations
  20. 20. Partners View more: http://frictionlessdata.io/partners/ Dataship YOUR RESEARCH ORG
  21. 21. ● Project website: http://frictionlessdata.io/ ● Specifications: http://specs.frictionlessdata.io/ ● GitHub: https://github.com/frictionlessdata/ ● User Stories: http://frictionlessdata.io/user-stories/
  22. 22. ● Newsletter: http://frictionlessdata.io/get- involved/#newsletter ● Follow @okfnlabs on Twitter (#frictionlessdata)

×