SlideShare a Scribd company logo
1 of 16
The Federated Data System DataFed: Experiences in Data Homogenization and Networking ,[object Object],[object Object],Spring AGU, May 29, 2008, Ft. Lauderdale
DataFed in a Nutshell: A Federation of  autonomous, distributed  data providers Performs non-intrusive  wrapping   of data into web services Provides service-based  analysis services  and tools General Experience with DataFed: It is an  agile  virtual data system can deliver  info products to diverse users Third-party  mediation can homogenize  distributed data on the fly Since 2005,  DataFed is used  by EPA and in research   DataFed Motivated by GEOSS DataFed development is guided by the meme of GEOSS
Five practices for agile, seamless data federation: ,[object Object],[object Object],[object Object],[object Object],[object Object]
Parameter-Space-Time Query  Using OGC WCS Data Access Protocol ,[object Object],[object Object],[object Object],Coverage=THEEDDS.T&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF Coverage=SEAW.Refl&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=GeoTIFF Coverage=SURF.Bext&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF-table Grid Image Station Data Parameter Bounding Box Time Range Out Format
[object Object],Third Party Data Wrappers   Heterogeneous input data  >>> Homogeneous (WCS) Query
Mediated User-Data Interface Mediator turns data into Views Mediated Integration  is a flexible design pattern for System of Systems   Client-Server  design is demanding: User carries the burden of integration Query Data Views
SOAP RDF Mashup Workflow Mashups: Loose Coupling of Autonomous Applications DataFed – Wiki -- GoogleEarth
DataSpaces for Datasets GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests invokes references publishes provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog Catalog User Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and  Users Shared Metadata by the Users, for the Users   Services Community DataSpaces
GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests invokes references publishes provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and  Users Shared Metadata by the Users, for the Users   views report Services Community DataSpaces
-+ GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests invokes references publishes provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog Service Workflow composes Data Analyst visualizes Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and  Users Shared Metadata by the Users, for the Users   views report Services Community DataSpaces
Wiki ‘DataSpaces’ Creating and Sharing Metadata Community Catalog - Find Dataset Describe Dataset Discuss Dataset ESIP Communal Wiki ,[object Object],[object Object],[object Object]
Sharing Best Practices: GEO Best Practice Wiki
Developments and Challenges: ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
ESIP Coordination Application
 
 

More Related Content

More from Rudolf Husar

111018 geo sif_aq_interop
111018 geo sif_aq_interop111018 geo sif_aq_interop
111018 geo sif_aq_interop
Rudolf Husar
 
110823 solta11 intro
110823 solta11 intro110823 solta11 intro
110823 solta11 intro
Rudolf Husar
 
110823 data fed_solta11
110823 data fed_solta11110823 data fed_solta11
110823 data fed_solta11
Rudolf Husar
 
110510 aq co_p_network
110510 aq co_p_network110510 aq co_p_network
110510 aq co_p_network
Rudolf Husar
 
110509 aq co_p_solta
110509 aq co_p_solta110509 aq co_p_solta
110509 aq co_p_solta
Rudolf Husar
 
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
Rudolf Husar
 
110410 aq user_req_methodology_sydney_subm
110410 aq user_req_methodology_sydney_subm110410 aq user_req_methodology_sydney_subm
110410 aq user_req_methodology_sydney_subm
Rudolf Husar
 
110408 aq co_p_uic_sydney_husar
110408 aq co_p_uic_sydney_husar110408 aq co_p_uic_sydney_husar
110408 aq co_p_uic_sydney_husar
Rudolf Husar
 
110105 htap pilot_aqco_p_esip_dc
110105 htap pilot_aqco_p_esip_dc110105 htap pilot_aqco_p_esip_dc
110105 htap pilot_aqco_p_esip_dc
Rudolf Husar
 
100615 htap network_brussels
100615 htap network_brussels100615 htap network_brussels
100615 htap network_brussels
Rudolf Husar
 
121117 eedss briefing_nasa_epa
121117 eedss briefing_nasa_epa121117 eedss briefing_nasa_epa
121117 eedss briefing_nasa_epa
Rudolf Husar
 
120910 nasa satellite_outline
120910 nasa satellite_outline120910 nasa satellite_outline
120910 nasa satellite_outline
Rudolf Husar
 
120612 geia closure_ofeo_ms_soa_subm
120612 geia closure_ofeo_ms_soa_subm120612 geia closure_ofeo_ms_soa_subm
120612 geia closure_ofeo_ms_soa_subm
Rudolf Husar
 
110414 extreme dustsmokesulfate
110414 extreme dustsmokesulfate110414 extreme dustsmokesulfate
110414 extreme dustsmokesulfate
Rudolf Husar
 
Aq Gci Infrastructure
Aq Gci InfrastructureAq Gci Infrastructure
Aq Gci Infrastructure
Rudolf Husar
 
AQ GCI Infrastructure
AQ GCI InfrastructureAQ GCI Infrastructure
AQ GCI Infrastructure
Rudolf Husar
 
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
Rudolf Husar
 
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
Rudolf Husar
 
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
Rudolf Husar
 
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
Rudolf Husar
 

More from Rudolf Husar (20)

111018 geo sif_aq_interop
111018 geo sif_aq_interop111018 geo sif_aq_interop
111018 geo sif_aq_interop
 
110823 solta11 intro
110823 solta11 intro110823 solta11 intro
110823 solta11 intro
 
110823 data fed_solta11
110823 data fed_solta11110823 data fed_solta11
110823 data fed_solta11
 
110510 aq co_p_network
110510 aq co_p_network110510 aq co_p_network
110510 aq co_p_network
 
110509 aq co_p_solta
110509 aq co_p_solta110509 aq co_p_solta
110509 aq co_p_solta
 
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
 
110410 aq user_req_methodology_sydney_subm
110410 aq user_req_methodology_sydney_subm110410 aq user_req_methodology_sydney_subm
110410 aq user_req_methodology_sydney_subm
 
110408 aq co_p_uic_sydney_husar
110408 aq co_p_uic_sydney_husar110408 aq co_p_uic_sydney_husar
110408 aq co_p_uic_sydney_husar
 
110105 htap pilot_aqco_p_esip_dc
110105 htap pilot_aqco_p_esip_dc110105 htap pilot_aqco_p_esip_dc
110105 htap pilot_aqco_p_esip_dc
 
100615 htap network_brussels
100615 htap network_brussels100615 htap network_brussels
100615 htap network_brussels
 
121117 eedss briefing_nasa_epa
121117 eedss briefing_nasa_epa121117 eedss briefing_nasa_epa
121117 eedss briefing_nasa_epa
 
120910 nasa satellite_outline
120910 nasa satellite_outline120910 nasa satellite_outline
120910 nasa satellite_outline
 
120612 geia closure_ofeo_ms_soa_subm
120612 geia closure_ofeo_ms_soa_subm120612 geia closure_ofeo_ms_soa_subm
120612 geia closure_ofeo_ms_soa_subm
 
110414 extreme dustsmokesulfate
110414 extreme dustsmokesulfate110414 extreme dustsmokesulfate
110414 extreme dustsmokesulfate
 
Aq Gci Infrastructure
Aq Gci InfrastructureAq Gci Infrastructure
Aq Gci Infrastructure
 
AQ GCI Infrastructure
AQ GCI InfrastructureAQ GCI Infrastructure
AQ GCI Infrastructure
 
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
 
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
 
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
 
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
 

Recently uploaded

Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
FIDO Alliance
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
FIDO Alliance
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 

2008-05-27: Spring AGU, DataFed Best Practices

  • 1.
  • 2. DataFed in a Nutshell: A Federation of autonomous, distributed data providers Performs non-intrusive wrapping of data into web services Provides service-based analysis services and tools General Experience with DataFed: It is an agile virtual data system can deliver info products to diverse users Third-party mediation can homogenize distributed data on the fly Since 2005, DataFed is used by EPA and in research DataFed Motivated by GEOSS DataFed development is guided by the meme of GEOSS
  • 3.
  • 4.
  • 5.
  • 6. Mediated User-Data Interface Mediator turns data into Views Mediated Integration is a flexible design pattern for System of Systems Client-Server design is demanding: User carries the burden of integration Query Data Views
  • 7. SOAP RDF Mashup Workflow Mashups: Loose Coupling of Autonomous Applications DataFed – Wiki -- GoogleEarth
  • 8. DataSpaces for Datasets GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests invokes references publishes provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog Catalog User Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and Users Shared Metadata by the Users, for the Users Services Community DataSpaces
  • 9. GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests invokes references publishes provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and Users Shared Metadata by the Users, for the Users views report Services Community DataSpaces
  • 10. -+ GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests invokes references publishes provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog Service Workflow composes Data Analyst visualizes Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and Users Shared Metadata by the Users, for the Users views report Services Community DataSpaces
  • 11.
  • 12. Sharing Best Practices: GEO Best Practice Wiki
  • 13.
  • 15.  
  • 16.  

Editor's Notes

  1. This talk is about the federated.. In particular our experineces with datafed over past 4 years of data homog and networking