Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TUW - 184.742 Data marketplaces: models and concepts

507 views

Published on

This presentation is part of the course "184.742 Advanced Services Engineering" at The Vienna University of Technology, in Winter Semester 2012. Check the course at: http://www.infosys.tuwien.ac.at/teaching/courses/ase/

Published in: Education
  • Be the first to comment

  • Be the first to like this

TUW - 184.742 Data marketplaces: models and concepts

  1. 1. Advanced Services Engineering, WS 2012, Lecture 6Data marketplaces: models and concepts Hong-Linh Truong Distributed Systems Group, Vienna University of Technology truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truongASE WS 2012 1
  2. 2. Outline Data marketplaces Description models Exchange data agreement Data contractASE WS 2012 2
  3. 3. Recall – data service units in clouds/internet data data data Data service unit Data service unit Data service unit data People Things Internet/CloudASE WS 2012 3
  4. 4. Recall – data as a service Data-as-a-Service – service models Data publish/subcription Database-as-a-Service middleware as a service (Structured/non-structured querying systems) Sensor-as-a-Service Storage-as-a-Service (Basic storage functions) deploy Private/Public/Hybrid/Community CloudsASE WS 2012 4
  5. 5. Data marketplaces More than just DaaS  DaaS focuses on data provisioning features Data marketplaces  Multiple data providers and consumers  Multiple DaaS  Complex interactions among DaaS, data providers and consumers  Complex billing and pricing models  Market dynamicsASE WS 2012 5
  6. 6. Discussion time WHAT ARE IMPORTANT ISSUES IN DATA MARKETPLACES?ASE WS 2012 6
  7. 7. Some important issues DAAS DESCRIPTION MODEL DATA AGREEMENT EXCHANGE DATA CONTRACTASE WS 2012 7
  8. 8. Description Model for DaaS (1) State of the art:  Providers have their own way to describe DaaS, mainly in HTML  Existing service description techniques are not adequate in supporting description for DaaS Problems  Service and data discovery cannot be done automatically  On-demand data integration, service integration, and query optimization cannot be supported well.  Service/data information and DaaS engineering cannot be tied.ASE WS 2012 8
  9. 9. Description Model for DaaS (2)Which levels must be covered? Here Data resource Data items Consumer Data Data Data items items assets Consumer Data resource Data resource Data resource Data resource DaaSASE WS 2012 9
  10. 10. Description Model for DaaS – types of informationWhich types of information must be covered? Quality of Ownership data Price License .... Service interface Service license Quality of service ....ASE WS 2012 10
  11. 11. DEMOS – a description model for Data-as-a-ServiceQuang Hieu Vu, Tran Vu Pham, HongLinh Truong,, Schahram Dustdar,Rasool Asal: DEMODS: A DescriptionModel for Data-as-a-Service. AINA2012: 605-612See prototype:http://www.infosys.tuwien.ac.at/prototype/SOD1/demods/ ASE WS 2012 11
  12. 12. Description model and data marketplacesASE WS 2012 12
  13. 13. DEMODS – prototype (1)ASE WS 2012 13
  14. 14. DEMODS – prototype (2) Check: http://demodsmanagement.appspot.com/ASE WS 2012 14
  15. 15. Discussion time WHICH TYPES OF DAAS INFORMATION ARE DYNAMIC? AND THEIR IMPACT ON DESCRIPTION MODELS?ASE WS 2012 15
  16. 16. Exchange data agreement (1)Consumer DaaS provider Data provider DaaS Consumer DaaS DaaS Sensor How they interact w.r.t. data concerns? How their data agreements look like?ASE WS 2012 16
  17. 17. Exchange data agreement (2)  Lack of models and protocols for data agreement in data marketplaces  Constraints for data usage are not clear  Inadequate data/service description → hindering data selection and integration  Existing techniques are not adequate for dynamic data agreement exchange in data marketplacesNeed generic exchange models suitable for differentways of data provisioning in data marketplaces ASE WS 2012 17
  18. 18. Data Agreement Exchange as a Service (DAES) Metamodel for data agreement exchange Techniques for enriching and associating data assets with agreement terms Interaction models for data agreement exchangeHong Linh Truong, Schahram Dustdar, Joachim Götze, Tino Fleuren, Paul Müller, Salah-Eddine Tbahriti, Michael Mrissa,Chirine Ghedira: Exchanging Data Agreements in the DaaS Model. APSCC 2011: 153-160ASE WS 2012 18
  19. 19. Metamodel for data agreements Different category of agreements  Licensing, privacy, quality of data Extensions  Languages  Different types of agreements  Different specifications ASE WS 2012 19
  20. 20. Associating data with data agreements Solutions  (a) directly inserting agreements into data assets  (b) providing two-step access to agreements and data assets  (c) linking data agreements to the description of DaaS  (d) linking data agreements to the message sent by DaaSASE WS 2012 20
  21. 21. Possible interaction models for data enriched with data agreementsASE WS 2012 21
  22. 22. DAES – conceptual architecture Jersey, JAX-RS Restful WS Weblogic Using URIs to identify agreementsASE WS 2012 22
  23. 23. DAES – managed information Specific applications: agreement creation, agreement validation, agreement compatibility analysis, agreement management Implementation: Jersey, JAX-RS Restful WS WeblogicASE WS 2012 23
  24. 24. Illustrating examples – insert agreement into data asset  A pay-per-use consumer uses dataAPI of DaaS search for data  The consumer pays the use APIs  Each call can return different types of data Example with People Search in InfochimpsBut a strong consequencefor data service engineeringtechniques: dealing withelastic requirements! ASE WS 2012 24
  25. 25. Illustrating examples – link agreements to geospatial data Domain-specific DaaS: different agreements for different data requests  Vector data of geographic features via Web-Feature-Service (WFS)  Terrain elevation data via Web-Coverage Services (WCS)ASE WS 2012 25
  26. 26. Illustrating examples – link agreements to geospatial data Consumers can interpret and reason if the data can be used for specific purposesASE WS 2012 26
  27. 27. Illustrative examples – develop an app for policy compliance (1)ASE WS 2012 27
  28. 28. Illustrative examples – develop an app for policy compliance (2)Configuration Results ASE WS 2012 28
  29. 29. Discussion time HOW NEAR-REALTIME DATA IMPACTS ON DATA AGREEMENT EXCHANGE?ASE WS 2012 29
  30. 30. Data contractHow to specific data contract? Data resource Data items Consumer Data Data Data items items assets Consumer Data resource Data resource Data resource Data resource DaaSASE WS 2012 30
  31. 31. Data contracts Give a clear information about data usage Have a remedy against the consumer where the circumstances are such that the acts complained of do not Limit the liability of data providers in case of failure of the provided data; Specify information on data delivery, acceptance, and paymentASE WS 2012 31
  32. 32. Data contracts  Well-researched contracts for services but not for DaaS and data marketplaces  But service APIs != data APIs =! data assets  Several open questions  Right to use data? Quality of data in the data agreement? Search based on data contract? Etc. ➔ Require extensible models ➔ Capture contractual terms for data contracts ➔ Support (semi-)automatic data service/data selection techniques.Hong-Linh Truong, Marco Comerio, Flavio De Paoli, G.R. Gangadharan, Schahram Dustdar, "Data Contracts forCloud-based Data Marketplaces ", International Journal of Computational Science and Engineering, 2012 Vol.7, No.4,pp.280 - 295 ASE WS 2012 32
  33. 33. Study of main data contract terms Data rights  Derivation, Collection, Reproduction, Attribution Quality of Data (QoD)  Not mentioned, Not clear how to establish QoD metrics Regulatory Compliance  Sarbanes-Oxley, EU data protection directive, etc. Pricing model  Different models, pricing for data APIs and for data assets Control and Relationship  Evolution terms, support terms, limitation of liability, etc Most information is in human-readable formASE WS 2012 33
  34. 34. Data contract studyASE WS 2012 34
  35. 35. Developing data contracts in cloud- based data marketplaces Follow community-based approach for data contract Propose generic structures to represent data contract terms and abstract data contracts Develop frameworks for data contract applications Incorporate data contracts into data-as-a-service description Develop data contract applications ASE WS 2012 35
  36. 36. Community view on data contract development Community users can develop:  Term categories, term names, values, and units  Rules for data contracts  Common contract and contract fragments Community users =! novice usersASE WS 2012 36
  37. 37. Representing data contract terms Contract term: (termName,termValue)  Term name: common terms or user-specific terms  Term value: a single value, a set, or a rangeASE WS 2012 37
  38. 38. Structuring abstract data contractsConcrete data contracts generatescan be in RDF, XML orJSON Use Identifiers and Tags for identifying and searches ASE WS 2012 38
  39. 39. Development of contract applications Main applications:  Data contract compatibility evaluation, data contract composition Some common steps  Extract DCTermType in TermCategoryType  Extact comprable terms from all contracts, - e.g., dataRight: Derivation, Composition and Reproduction  Use evaluation rules associated with DCTermType from from rule repositories  Execute rules by passing comparable terms to rules  Aggregate resultsASE WS 2012 39
  40. 40. Prototype RDF for representing term categories, term names, term values, units Allegro Graph for storing contract knowledgeASE WS 2012 40
  41. 41. Illustrating examples A large sustainability monitoring data platform shows how green buildings are  Real-time total and per capita of CO2 emission of monitored building  Open government data about CO2 per capita at national level We created contracts from  Open Data Commons Attribution License  Open Government License ASE WS 2012 41
  42. 42. Existingcommonknowledgeabout OpenDataCommons ASE WS 2012 42
  43. 43. Step 2: provide OpenBuildingCO2OpenBuildingCO2 by OpenGov formodifying quality of government datadata and data right Data contract for green building data ASE WS 2012 43
  44. 44. Experiments – composing data contract termsASE WS 2012 44
  45. 45. Discussion time CAN WE AUTOMATICALLY GENERATE DATA CONTRACTS FOR NEAR-REALTIME DATA?ASE WS 2012 45
  46. 46. Exercises Read mentioned papers Examine existing data marketplaces and write DEMODS-based specification for some of them Develop some specific data contracts for open government data Work on some algorithms for checking data contract compatiblityASE WS 2012 46
  47. 47. Thanks for your attention Hong-Linh Truong Distributed Systems Group Vienna University of Technology truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truongASE WS 2012 47

×