Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Industry@RuleML2015 DataGraft

486 views

Published on

One stop shop for hosted data management

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Industry@RuleML2015 DataGraft

  1. 1. DataGraft Dumitru Roman dumitru.roman@sintef.no @RuleML2015 http://datagraft.net
  2. 2. Outline • What is DataGraft • Case study example: PLUQI • DataGraft Rationale 2
  3. 3. Developed to allow data workers to manage their data in a simple, effective, and efficient way Powerful data transformation and scalable data access capabilities 3
  4. 4. One stop shop for hosted data management • Interactively build, modify and share data transformations • Reuse transformations to repeatably clean and transform spreadsheet data • Host and share data and transformations in cloud based catalog 4
  5. 5. Transformation pipeline with data preview 5
  6. 6. Catalogue of public data pages and transformations
  7. 7. Flexible management and sharing of data and transformations • Choose to share transformations or data pages privately or publicly • Fork, reuse and extend transformations built by other professionals from DataGraft public catalog • API access to public or private datasets and transformations 7
  8. 8. Public/Private and sharing of transformations
  9. 9. Reusing transformations
  10. 10. APIs
  11. 11. Reliable data hosting and querying services • Query Join and visualise datasets through own SPARQL endpoint • Scalable RDF data hosting • Visualise data with SPARQL based chart builder 11
  12. 12. Data page 12
  13. 13. Data visualization 13
  14. 14. Case study: PLUQI 14 PLUQI: Personalized and Localized Urban Quality Index isA Application (mobile/Web) showing a customizable index that represents and visualize the level of well-being and sustainability for given cities based on individual preferences. The index model includes various domains: Daily life satisfaction: weather, transportation, community etc.; Healthcare level: number of doctors, hospitals, suicide statistics, etc.; Safety and security: number of police stations, fire stations, crimes per capita, etc.; Financial satisfaction: prices, incomes, housing, savings, debt, insurance, pension, etc.; Level of opportunity: jobs, unemployment, education, re-education, economic dynamics, etc.; Environmental needs and efficiency: green space, air quality, etc.;
  15. 15. Example tabular data transformation
  16. 16. 16
  17. 17. 17
  18. 18. 18
  19. 19. 19
  20. 20. 20
  21. 21. 21
  22. 22. 22
  23. 23. 23
  24. 24. 24
  25. 25. 25
  26. 26. 26
  27. 27. 27
  28. 28. 28
  29. 29. 29
  30. 30. 30
  31. 31. 31
  32. 32. 32
  33. 33. 33
  34. 34. 34
  35. 35. 35
  36. 36. 36
  37. 37. 37
  38. 38. Interactive transformation of PLUQI data
  39. 39. Pipeline – high-level generated code (Clojure with Grafter library) 39
  40. 40. Specifying RDF mappings
  41. 41. RDF mappings – high-level generated code (Clojure with Grafter library) 41
  42. 42. PLUQI – potential usage • Place recommendation for travel agencies or travelers • Policy analysis and optimization for (local) government • Understanding the citizen’s voice and demands regarding environmental conservation • Commercial impact analysis for retailer and franchises • Location recommendation and understanding local issues for real estate • Risk analysis and management for insurance and financial companies • Local marketing and sales force optimization for marketers 42
  43. 43. DataGraft Rationale 43
  44. 44. Open Data • Businesses can develop new ideas, services and applications; improve decision making, cost savings • Can increase government transparency and accountability, quality of public services • Citizens get better and timely access to public services
  45. 45. Lots of open datasets on the Web… • A large number of datasets have been published as open data in the recent years • Many kinds of data: cultural, science, finance, statistics, transport, environment, … • Popular formats: tabular (e.g. CSV, XLS), HTML, XML, JSON, … 45
  46. 46. …but few actually used • Few applications utilizing open and distributed datasets at present • Challenges for data consumers – Data quality issues – Difficult or unreliable data access – Licensing issues • Challenges for data publishers – Lack of expertise & resources: not easily to publish & maintain high quality data – Unclear monetization & sustainability 46 Open Data Portal Datasets Applications data.gov ~ 110 000 ~ 350 publicdata.eu ~ 50 000 ~ 80 data.gov.uk ~ 20 000 ~ 350 data.norge.no ~ 300 ~ 40
  47. 47. Open Data is mostly tabular data – Records organized in silos of collections – Very few links within and/or across collections – Difficult to understand the nature of the data – Difficult to integrate / query 47 Tabular datasets publicdata.eu data.gov.uk
  48. 48. Linked Data • Method for publishing data on the Web • Self-describing data and relations • Interlinking • Accessed using semantic queries • A set of standards developed by W3C – Data format: RDF – Knowledge representation: RDFS/OWL – Query language: SPARQL – Linking medium: HTTP http://www.w3.org/standards/semanticweb/data 48
  49. 49. Linked Open Data Cloud 49
  50. 50. Example 50
  51. 51. Linked Data is great for Open Data • Linked Data as a great means to represent and integrate disparate and heterogeneous open data sources • How Linked Data can improve Open Data: – Easier integration, free data from silos – Seamless interlinking of data – Understand the data – New ways to query and interact with data • Challenges with using Linked Data – Lack of tooling & expertise to publish high quality Linked Data – Lack of resources to host LOD endpoints / unreliable data access 51
  52. 52. DataGraft: Packaging Linked Data • Packaging Linked Data to make it more approachable to the open data community – Publishers – Developers – Data workers 52
  53. 53. 53 DataGraft – 1 package 2 audiences DataGraft Data Publisher Application Developer Helping publishing open data Giving better, easier tools
  54. 54. 54 Tabular Data Graph Data
  55. 55. Data Transformation: Data + Transformation 55 Raw Data Transformation process Usable Data Transformation Reuse Share Evolve …
  56. 56. DataGraft means to making Open (Linked) Data easier to use • A platform/hosting: to make it easy for publishers to put data on the web, and developers to publish their applications • A portal: to help advertising data and applications availability - and enticing new users • Tool-supported data transformation methodology: to make it easy for people with Excel knowledge to publish large amounts of high quality data • API's with high-quality documentation: for processing large amounts of data reliably in order to create interactives, visualisations and transformations 56 Make Linked Data more accessible to everyone!
  57. 57. DataGraft Enablers 57 Grafter Grafterizer RDF DBaaSData Portal DataGraft
  58. 58. DataGraft – targeted impacts 58 • A reduction in the cost for organisations (e.g. SMEs, public organizations, etc) which lack sufficient expertise and resources to publish open data • A reduction on the dependency of open data publishers on generic Cloud platforms to build, deploy and maintain their open/linked data from scratch • An increase in the speed of publishing new datasets and updating existing datasets
  59. 59. DataGraft– targeted impacts (cont’) 59 • A reduction in the cost and complexity of developing applications that use open data • An increase in the reuse of open data by providing fast and seamless access to numerous open data sets to the applications hosted on DataGraft.net
  60. 60. Summary 60 • DataGraft – emerging solution (as-a-Service) for making Open (Linked) Data more accessible – Platform, portal, methodology, APIs • Key features: – Support for Sharable/Repeatable/Reusable Data Transformations – Reliable RDF Database-as-a-Service
  61. 61. 61
  62. 62. DataGraft Data-as-a-Service tools brought to you by 62 http://dapaas.eu
  63. 63. http://datagraft.net @dapaasproject dapaas-platform@googlegroups.com Thank you! 63 Contact: dumitru.roman@sintef.no

×