Curation and Characterization of Web Services

241 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
241
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Curation and Characterization of Web Services

  1. 1. Curation and Characterization of Web Services Jose Enrique Ruiz jer@iaa.es October 23rd 2012 2012 IVOA Fall Interop Meeting - Sao Paolo 1
  2. 2. Wf4Ever E-SCIENCEWf4Ever 2011 - 2013Advanced Workflow Preservation Technologies for Enhanced Science 1.  Intelligent Software Components (ISOCO, Spain) 2.  University of Manchester (UNIMAN, UK) 2 7 3.  Universidad Politécnica de Madrid (UPM, Spain) 5 4 4.  Poznan Supercomputing and Networking Centre (Poland) 5.  University of Oxford and OeRC (OXF, UK) 1 6.  Instituto Astrofísica Andalucía (IAA-CSIC, Spain) 3 6 7.  Leiden University Medical Centre (LUMC, NL) 2
  3. 3. WS ProvidersThe next generation of archivesMuch wider FoV and spectral coverage•  Huge sized datasets (~ tens TB)•  Big Data science highly dependent on I/O data rates•  Subproducts as virtual data generated on-the-flyWe are moving into a world where•  computing and storage are cheap•  data movement is death 3
  4. 4. WS ProvidersThe next generation of archivesMuch wider FoV and spectral coverage•  Huge sized datasets (~ tens TB)•  Big Data science highly dependent on I/O data rates•  Subproducts as virtual data generated on-the-flyThe move computing to data paradigmArchives should evolve from data providers intoservices providers, where web services may help tosolve bandwidth issues. 4
  5. 5. WS ProvidersThe next generation of archivesMuch wider FoV and spectral coverage•  Huge sized datasets (~ tens TB)•  Big Data science highly dependent on I/O data rates•  Subproducts as virtual data generated on-the-flyData DiscoveryData AccessData Management 5
  6. 6. WS ProvidersThe next generation of archivesMuch wider FoV and spectral coverage•  Huge sized datasets (~ tens TB)•  Big Data science highly dependent on I/O data rates•  Subproducts as virtual data generated on-the-flyWeb Services DiscoveryWeb Services AccessWeb Services Management 6
  7. 7. WS vs. DataThe Data Lifecycle Credited Published Used Discovered Transformed 7
  8. 8. WS vs. DataThe Service Lifecycle Credited Published Used Discovered Transformed 8
  9. 9. WS vs. DataPublished•  The VO Registry•  Easier to publish services than datasets in the VO ?•  WS are not exclusive property of big data archives•  Publication is not Preservation•  Backup strategies•  Replication/Mirrors•  Versioning•  Software Publishing Platforms 9
  10. 10. WS vs. DataDiscovered•  Search Criteria •  Relevant Keywords (Semantics) •  Authoring - Institution, Archive •  Waveband, Science •  Function-based •  VO Services mainly focused on Data Discovery and Access (DAL) •  Wrapped Legacy Apps and Data Processing (SIAv2, Theory IG) •  KDD IG •  Input/Output Data (TAP, UTypes, VOSI #tables) •  Access Policy (Authentication – SSO, OAuth) •  A-Synchrony (SOAP, REST) and Stage Data (VOSpace) •  Allocation of CPU/Storage, Estimated Computing Time•  Proposition of alternatives and similars 10
  11. 11. WS vs. DataUsed and Transformed•  How to use them ? (WADL, WSDL – VOSI #capabilities) •  Input Data -> Parameters needed and formats •  Self-described WS (PDL, S3, SimDAL, SimDB) •  Output Data -> Response format - TAP •  Example Data, Self-Consistency Checking•  Access Policy (Authentication – SSO, OAuth)•  WS orchestration in Workflows (Data-flow vs. Control-flow)•  How the community uses WS ?•  Propositions based on patterns of statistical use or popularity•  Provenance of the methods is Wf-evolution by re-use•  Consumed by Humans and Machines - Interoperable (WS-I) 11
  12. 12. WS vs. DataCredited•  Linked to related Artefacts •  Data Facilities and Archives •  Authors, ACSL Software, Wfs•  Quality Assessment •  Technical and scientific •  Penalize abandoned and award the maintained•  Automate Monitoring (VOSI #availability) •  Decay •  Performance, WS Analytics •  Change of interfaces, permissions, etc.•  Community Curation •  Blogging •  Recommendation •  Folksonomy 12
  13. 13. ConclusionsIn a cloud of web services and data..Web Services should benefit of thesame privileges acquired by Data until now.Start thinking on how to provide•  Detailed curation•  Thorough characterization 13

×