050317 Ws Telecon Husar


Published on


Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • AQ data arise from diverse sources, each having specific history, driving forces, formats, quality, etc. Data analysis, i.e. turning the raw data into ‘actionable’ knowledge, requires combining data from these sources The three major data ‘processing’ operations (services) are filtering, aggregation and fusion
  • 050317 Ws Telecon Husar

    1. 1. Web Services: ES Rationale and Assertions <ul><li>Prepared for: </li></ul><ul><li>Technology Infusion Web Services Sub-group </li></ul><ul><li>March 17, 2004 Telecon </li></ul><ul><li>R. Husar, rhusar@me.wustl.edu </li></ul>Provider Push Science Pull Flow of Data Flow of Control DATA 1 Data 2 Data 2 Knowledge Prod. 1 Knowledge Prod. 2 Knowledge Prod. 4 Knowledge Prod. 3 Knowledge Prod. 5 Web Services for Refining Data to Knowledge
    2. 2. [Better Earth] Science is the DRIVER for the Information System!
    3. 3. The Researcher’s Challenge “ The researcher cannot get access to the data; if he can, he cannot read them; if he can read them, he does not know how good they are; and if he finds them good he cannot merge them with other data.” Information Technology and the Conduct of Research: The Users View National Academy Press, 1989 <ul><li>These resistances can be overcome through </li></ul><ul><li>A catalog of distributed data resources for easy data ‘ discovery ’ </li></ul><ul><li>Uniform data coding and formatting for easy access, transfer and merging </li></ul><ul><li>Rich and flexible metadata structure to encode the knowledge about data </li></ul><ul><li>Powerful shared tools to access, merge and analyze the data </li></ul>
    4. 4. [For the data types they cover], OGC & OpenDAP are addressing the Finding and Reformatting tasks The custom processing of data into knowledge is still a major burden at the user end
    5. 5. Earth Science Data to Knowledge Transformation: Value-Adding Processes Petabytes 10 15 Terabytes 10 12 Gigabytes 10 9 Megabytes 10 6 Calibration, Transformation To Characterized Geophysical Parameters Filtering, Aggregation, Fusion, Modeling, Trends, Forecasting Interactive Dissemination ACCESS Multi-platform/parameter, high space/time resolution, remote & in-situ sensing Sensing Analysis & Synthesis Data Acquisition Value Chain (Network) InfoSystem Goal: Add as much value to the data as possible to benefit all users Data Usage Value Network Flexible data selection, and processing to to deliver right knowledge, right place right time Data - L1 Information – L2 Knowledge – L3-6? Usable Knowledge Query Data Distributed, Dynamic More Local, DAAC Processing Knowledge Use
    6. 6. Value-Added Processing in Service Oriented Architecture <ul><li>Peer-to-peer network representation </li></ul>Data, services and users are distributed throughout the network Users compose data processing chains form reusable services Intermediate data are also exposed for possible further use Chains can be linked to form compound value-adding processes Service chain representation User Tasks: Fi nd data and services Compose service chains Expose output Chain 2 Chain 1 Chain 3 Data Service User Carries less Burden In service-oriented peer-to peer architecture, the user is aided by software ‘agents’ Control Data Chain 1 Chain 2 Chain 3 Data Service Catalog User
    7. 7. Data Flow and Flow Control in AQ Management <ul><li>Data are supplied by the provider and exposed on the ‘smorgasbord’ </li></ul><ul><li>However, the choice of data and processes is made by the user </li></ul><ul><li>Thus, the autonomous data consumers, providers and mediators form the info system </li></ul>Provider Push User Pull Flow of Data Flow of Control AQ DATA METEOROLOGY EMISSIONS DATA Informing Public AQ Compliance Status and Trends Network Assess. Tracking Progress Data to Knowledge ‘Refinery’ <ul><li>The data ‘refining’ process is not a chain but network connection processing nodes. </li></ul><ul><li>Like on the Internet, new nodes and connections are added continuously </li></ul><ul><li>Thus, the infosystem needs to support the dynamic addition of new nodes and connections </li></ul><ul><li>Hence – there is a need for loosely coupled ‘plug-and-play’ architecture </li></ul>
    8. 9. A Sample of Datasets Accessible through DataFed/ESIP Mediation Near Real Time (~ day) <ul><li>It has been demonstrated (project FASTNET) that these and other datasets can be accessed, repackaged and delivered by AIRNow through ‘Consoles’ </li></ul>MODIS Reflectance MODIS AOT TOMS Index GOES AOT GOES 1km Reflec NEXTRAD Radar MODIS Fire Pix NRL MODEL NWS Surf Wind, Bext
    9. 10. Assertions on Web Services Technology <ul><li>Currently Web Services are the leading (and only?) technologies for building software applications in autonomous, networked, dynamic environment </li></ul><ul><li>The future is promising since businesses are driving the WS technologies and the community is benefiting from the increasingly ‘semantic web’ </li></ul><ul><li>A growing resource pool is exposed as ‘services’ and WS-based ES applications development frameworks are being developed/evaluated (e.g. SciFlo, DataFed) </li></ul><ul><li>WS Adaptation Issues </li></ul><ul><li>Catalogs for finding and using services are grossly inadequate </li></ul><ul><li>The semantic layers of the interoperability stack are not yet available </li></ul><ul><li>General ‘ fallacies of distributed computing’ : </li></ul><ul><ul><li>Network is reliable </li></ul></ul><ul><ul><li>Latency is zero </li></ul></ul><ul><ul><li>Bandwidth infinite </li></ul></ul><ul><ul><li>Network is secure </li></ul></ul><ul><ul><li>Topology stable </li></ul></ul><ul><ul><li>One administrator </li></ul></ul><ul><ul><li>No transport costs </li></ul></ul><ul><ul><li>Network uniform </li></ul></ul>
    10. 11. Interoperability Stack <ul><li>Kickoff Questions </li></ul><ul><li>What is a Web Service? </li></ul><ul><ul><li>e.g. 'A programming module with a well-defined, web-based I/O interface' (operating on well structured data??) </li></ul></ul><ul><ul><li>Examples of what is/is not a WS </li></ul></ul><ul><li>WS Classification by Interoperability Layer </li></ul><ul><ul><li>Transport </li></ul></ul><ul><ul><li>Interface Syntax </li></ul></ul><ul><ul><ul><li>Strongly typed interface (e.g. SOAP, WSDL) </li></ul></ul></ul><ul><ul><ul><li>Weakly typed interface (e.g. arbitrary CGI? URL interface) </li></ul></ul></ul><ul><ul><li>Protocol/Data </li></ul></ul><ul><ul><li>Semantics </li></ul></ul><ul><li>WS Classification by Architecture </li></ul><ul><ul><li>Services for Tightly Coupled applications (e.g. URL service called from IDL) </li></ul></ul><ul><ul><li>Services for Loosely Coupled (e.g. application composed from SOAP services) </li></ul></ul>HTTP, SMTP Addressing, Data flow Transport XML Data format Syntax SOAP, WS-* ext. Communication behavior Protocol Schema, WSDL Types Data WSDL ext., Policy, RDF Meaning Semantics Standards Description Layer