Your SlideShare is downloading. ×
0
Department of Civil, Architectural & Environmental Engineering<br />1<br />June 2, 2011<br />Development of a Hydrologic C...
Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model/Module<br />Data<br /...
Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model/Module<br />Data<br /...
Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model/Module<br />Data<br /...
Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model<br />Model/Module<br ...
Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model/Module<br />Data<br /...
Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Objective: Develop a Hydrol...
6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />2<br />Technical Issues<br /><ul><li>Mig...
 Lack of credibility of the algorithms or methods encapsulated in the codes
 Poor documentation of source codes
 Lack of “good coding practices”
Model Integration
 Lack of modular model structure
 Intertwining of user interfaces and computing kernels
 Incompatible programming languages
Data Interoperability
 Distinct input and output data structures of models
 Distinct data models undertaken by disparate data sources
 Data semantics</li></li></ul><li>6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />3<br /...
Facilities: tools that ease the development  of component models or the migration of legacy models. Data analysis tools, t...
Workbench: a platform for model linkage, execution and management, usually supports graphical, icon-based model constructi...
 Supporting high-performance computations and provenance capture.
 Programming background</li></li></ul><li>6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br ...
Will its run-time performance be affected when migrating a legacy model into the environment?
Will the developed hydrologic community modeling system be flexible to use? </li></li></ul><li>Department of Civil, Archit...
Department of Civil, Architectural & Environmental Engineering<br />6<br />What is TRIDENT?<br /><ul><li>A workflow engine...
Composing, executing, monitoring and recording workflows
Embedding and running workflows in Word documents</li></ul>invoke<br /><ul><li>Executing a workflow on its located server
Scheduling workflow execution
Loading/running workflows from local/remote database
Loading/running workflows from local/remote database
Loading/running workflows from local/remote database
Running multiple workflows on different nodes of a server cluster</li></ul>TRIDENT SQL DATABASE<br />6/2/2011<br />
Department of Civil, Architectural & Environmental Engineering<br />7<br />Why use TRIDENT in hydrologic modeling?  <br />...
6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />Introduction of the libraries of HCMS <b...
6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />9<br />1.Data Access Library<br />Data S...
6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />10<br />Get National Elevation Data(NED)...
 NLCD: 30m * 30m, GeoTIFF</li></ul>[Activity 1] — Access NED or NLCD data within a specified area via Application Services...
6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />10<br />Get National Elevation Data(NED)...
 NLCD: 30m * 30m, GeoTIFF</li></ul>[Activity 1] — Access NED or NLCD data within a specified area via Application Services...
6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />10<br />Get National Elevation Data(NED)...
 NLCD: 30m * 30m, GeoTIFF</li></ul>[Activity 1] — Access NED or NLCD data within a specified area via Application Services...
Get NASA Land Data Assimilation System(NLDAS-2) Data<br /><ul><li> National coverage, 0.125*0.125 degree (approximately 13...
 Temperature, Precipitation, Long wave/Short wave radiation, Pressure, Vertical/Horizontal wind speed etc.</li></ul>[Activ...
Get NASA Land Data Assimilation System(NLDAS-2) Data<br /><ul><li> National coverage, 0.125*0.125 degree (approximately 13...
 Temperature, Precipitation, Long wave/Short wave radiation, Pressure, Vertical/Horizontal wind speed etc.</li></ul>[Activ...
6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />12<br />Get NWS Multi-sensor Precipitati...
Department of Civil, Architectural & Environmental Engineering<br />13<br />HIS Central Metadata WS<br />WaterOneFlowWS<br...
 It facilitates retrieving hydrologic and meteorological observation time series data from a central metadata catalogue (H...
Get  Data via WaterOneFlow Web Services in TRIDENT<br />[Activity 1] — Get web services within a specified geospatial exte...
6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />15<br />Get SSURGO Soil Data & Get EPA <...
 Accessing National Hydrography Dataset( watershed and stream shapefile) via EPA Geospatial Services.</li></li></ul><li>De...
 Delineate watershed/sub-watershed boundary, Generate river network; Create Triangulated Irregular Network(TIN); Process S...
Time Series Processing
 Interpolation/Extrapolation, Unit Conversion.
Data processing customized for data sources
Upcoming SlideShare
Loading in...5
×

Presentation for slideshare

404

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
404
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Sce-ua: Shuffled Complex Evolution-University of Arizona
  • Transcript of "Presentation for slideshare"

    1. 1. Department of Civil, Architectural & Environmental Engineering<br />1<br />June 2, 2011<br />Development of a Hydrologic Community Modeling System Using a Workflow Engine<br />Committee<br />BO LU<br />Dr. Michael Piasecki<br />Drexel University<br />Dr. Jonathan Goodall<br />Dr. Franco Montalto<br />Dr. Mira Olson<br />Dr. Ilya Zaslavsky<br />6/2/2011<br />
    2. 2. Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model/Module<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data/Data access<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tools of transformation, analysis, display etc.<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    3. 3. Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model/Module<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data/Data access<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tools of transformation, analysis, display etc.<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    4. 4. Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model/Module<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data/Data access<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tools of transformation, analysis, display etc.<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    5. 5. Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model<br />Model/Module<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data/Data access<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tools of transformation, analysis, display etc.<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    6. 6. Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Model/Module<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data/Data access<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tools of transformation, analysis, display etc.<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    7. 7. Department of Civil, Architectural & Environmental Engineering<br />1<br />Let’s imagine…<br />Objective: Develop a Hydrologic Community Modeling System(HCMS) that allows constructing seamlessly integrated hydrologic models with swappable and portable modules. <br />Model/Module<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data/Data access<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tools of transformation, analysis, display etc.<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    8. 8. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />2<br />Technical Issues<br /><ul><li>Migration of legacy models
    9. 9. Lack of credibility of the algorithms or methods encapsulated in the codes
    10. 10. Poor documentation of source codes
    11. 11. Lack of “good coding practices”
    12. 12. Model Integration
    13. 13. Lack of modular model structure
    14. 14. Intertwining of user interfaces and computing kernels
    15. 15. Incompatible programming languages
    16. 16. Data Interoperability
    17. 17. Distinct input and output data structures of models
    18. 18. Distinct data models undertaken by disparate data sources
    19. 19. Data semantics</li></li></ul><li>6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />3<br />What can facilitate this development? <br />Coupling Frameworksand Workflow Engines<br /><ul><li>Model standard or protocol: standard interfaces that component models should comply with, description of model structure, data model etc.
    20. 20. Facilities: tools that ease the development of component models or the migration of legacy models. Data analysis tools, transformation tools etc.
    21. 21. Workbench: a platform for model linkage, execution and management, usually supports graphical, icon-based model construction. </li></ul>Our choice: Microsoft’s TRIDENT workflow engine<br /><ul><li> How well a workflow engine can facilitate the development of community modeling system?
    22. 22. Supporting high-performance computations and provenance capture.
    23. 23. Programming background</li></li></ul><li>6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />4<br />Issues in development<br /><ul><li>Will the auxiliary platform be convenient to use? Does it involve a steep learning curve?
    24. 24. Will its run-time performance be affected when migrating a legacy model into the environment?
    25. 25. Will the developed hydrologic community modeling system be flexible to use? </li></li></ul><li>Department of Civil, Architectural & Environmental Engineering<br />5<br />Development of a Hydrologic Community Modeling System (HCMS) using TRIDENT workflow engine<br />TRIDENT<br />Analysis&UtilitiesLib.<br />Data Access Lib.<br />Model Performance Analysis: Water balance check, simulated vs. observed hydrograph comparison, performance statistics… <br />Get data from online repositories(CUAHSI HIS, NLDAS, MPE, USGS NED, NLCD, SSURGO etc.)<br />Get data from local repository, e.g. NetCDF, Excel, SQL Database…<br />Hydrologic Model Lib.<br />Data Process Lib.<br />Result Storage &Visualization<br />Evapotranspiration<br />Time series data processing: temporal interpolation, unit conversion…<br />Runoff Yield<br />Direct Runoff Routing<br />Base Flow<br />Geospatial data processing: watershed delineation, land coversoil data processing…<br />Channel Routing<br />SWAT<br />TOPMODEL<br />Seamless Integration<br />6/2/2011<br />
    26. 26. Department of Civil, Architectural & Environmental Engineering<br />6<br />What is TRIDENT?<br /><ul><li>A workflow engine that facilitates composing, executing, archiving and sharing scientific workflows. </li></ul>workflows(.twp)<br />Supported Services<br />Interactive Execution Service<br />workflows(.xoml)<br />Provenance Recording Service<br />Activities(.dll)<br />Schedule Execution Service <br />myExperiment website<br />Standard Classes<br />Workflows (.wfl)<br />Message Passing Service <br />Publish : workflows<br />Workflow Composer<br />WORD Add-in<br />ManagementStudio<br />Workflow Application<br /><ul><li>Managing workflows, activities, users, workflow provenance
    27. 27. Composing, executing, monitoring and recording workflows
    28. 28. Embedding and running workflows in Word documents</li></ul>invoke<br /><ul><li>Executing a workflow on its located server
    29. 29. Scheduling workflow execution
    30. 30. Loading/running workflows from local/remote database
    31. 31. Loading/running workflows from local/remote database
    32. 32. Loading/running workflows from local/remote database
    33. 33. Running multiple workflows on different nodes of a server cluster</li></ul>TRIDENT SQL DATABASE<br />6/2/2011<br />
    34. 34. Department of Civil, Architectural & Environmental Engineering<br />7<br />Why use TRIDENT in hydrologic modeling? <br />Composing workflows with swappable activities via the drag-and-drop manner on a GUI. <br />Flexible Model Setup<br />Allowing automatic and holistic execution without any external intervenes, or alternatively, interactive execution with the control of users.<br />Interactive/Non-interactive Execution<br />High-performance Computing<br />Allowing parallel or concurrent execution, distributed computations in the GRID environment.<br />Recording who, how, what and which resources are used in a workflow, and the derivation flow of data products. It ensures repeatability of model executions.<br />Provenance Capture<br />Easy to Share<br />Sharing workflow through publication mechanismsor repositories. <br />6/2/2011<br />
    35. 35. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />Introduction of the libraries of HCMS <br />Data Access Library<br />Data Processing Library<br />Hydrologic Model Library<br />Post-Anaylysis & Utilities Library<br />8<br />
    36. 36. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />9<br />1.Data Access Library<br />Data Sources:<br />Retrieving data from following data sources using SOAP/FTP protocols . <br />
    37. 37. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />10<br />Get National Elevation Data(NED), National Land Cover Data (NLCD)<br /><ul><li> NED: [1, 1/3, 1/9 arc second], [ ArcGrid, GeoTIFF, GridFloat, BIL]
    38. 38. NLCD: 30m * 30m, GeoTIFF</li></ul>[Activity 1] — Access NED or NLCD data within a specified area via Application Services. <br />[Activity 2] — Decompress downloaded data files. <br />
    39. 39. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />10<br />Get National Elevation Data(NED), National Land Cover Data (NLCD)<br /><ul><li> NED: [1, 1/3, 1/9 arc second], [ ArcGrid, GeoTIFF, GridFloat, BIL]
    40. 40. NLCD: 30m * 30m, GeoTIFF</li></ul>[Activity 1] — Access NED or NLCD data within a specified area via Application Services. <br />[Activity 2] — Decompress downloaded data files. <br />
    41. 41. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />10<br />Get National Elevation Data(NED), National Land Cover Data (NLCD)<br /><ul><li> NED: [1, 1/3, 1/9 arc second], [ ArcGrid, GeoTIFF, GridFloat, BIL]
    42. 42. NLCD: 30m * 30m, GeoTIFF</li></ul>[Activity 1] — Access NED or NLCD data within a specified area via Application Services. <br />[Activity 2] — Decompress downloaded data files. <br />
    43. 43. Get NASA Land Data Assimilation System(NLDAS-2) Data<br /><ul><li> National coverage, 0.125*0.125 degree (approximately 13.8km), 1979-present, 1-hour time interval.
    44. 44. Temperature, Precipitation, Long wave/Short wave radiation, Pressure, Vertical/Horizontal wind speed etc.</li></ul>[Activity 1] — Download hourly data files(GRIB) from NLDAS-2 data server. ftp://hydro1.sci.gsfc.nasa.gov/data/s4pa/NLDAS/NLDAS_FORA0125_H.002/<br />[Activity 2] — Make a choice of fields from a given field list, the activity then extracts data of selected fields from the downloaded data files via a decoder “WGRIB”. <br />[Activity 3] — Cut gridded data set within a specified geospatial extent. <br />6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />11<br />
    45. 45. Get NASA Land Data Assimilation System(NLDAS-2) Data<br /><ul><li> National coverage, 0.125*0.125 degree (approximately 13.8km), 1979-present, 1-hour time interval.
    46. 46. Temperature, Precipitation, Long wave/Short wave radiation, Pressure, Vertical/Horizontal wind speed etc.</li></ul>[Activity 1] — Download hourly data files(GRIB) from NLDAS-2 data server. ftp://hydro1.sci.gsfc.nasa.gov/data/s4pa/NLDAS/NLDAS_FORA0125_H.002/<br />[Activity 2] — Make a choice of fields from a given field list, the activity then extracts data of selected fields from the downloaded data files via a decoder “WGRIB”. <br />[Activity 3] — Cut gridded data set within a specified geospatial extent. <br />6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />11<br />
    47. 47. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />12<br />Get NWS Multi-sensor Precipitation Estimates (MPE)<br /><ul><li> National coverage, 4km*4km, 2005-present, 1-day time interval. </li></ul>[Activity 1] — Download 24-hour data files(NetCDF) from NWS MPE data server. http://water.weather.gov/precip/p_download_new/<br />[Activity 2] — Parse precipitation data from downloaded NetCDF files, and export them in the format of standard arrays.<br />[Activity 3] — Cut gridded data set within a specified geospatial extent. <br />
    48. 48. Department of Civil, Architectural & Environmental Engineering<br />13<br />HIS Central Metadata WS<br />WaterOneFlowWS<br />Get Data via WaterOneFlow web services<br /><ul><li> WaterOneFlow: a family of web services developed by CUAHSI using the SOAP protocol.
    49. 49. It facilitates retrieving hydrologic and meteorological observation time series data from a central metadata catalogue (HISCentral located at the San Diego Supercomputer Center) which holds the richest metadata information in the world for water data. </li></ul>Variable Name (e.g. precipitation)<br />Service ID (optional)<br />Geographical Extent (watershed boundary or latitude/longitude )<br />Temporal Extent<br />Get Web Services In Box<br />Semantic Checking…<br /> Get Sites<br />Web Service IDs<br />Updated Variables <br />Ontology Dictionary<br />Sites Metadata<br />Get Variables<br /> Verify Variable Catalog<br /> Get Time Series Data<br />Variable Codes<br />WaterML<br />Time Series Data/Metadata<br />Parse<br />Output<br />UI<br />Processing Step<br />Configuration Input<br />Web Service<br />6/2/2011<br />
    50. 50. Get Data via WaterOneFlow Web Services in TRIDENT<br />[Activity 1] — Get web services within a specified geospatial extent. <br />[Activity 2] — Get site and variable metadata based on given variable name. <br />[Activity 3] — Get time series data of given variable within the given geospatial extent.<br />6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />14<br />
    51. 51. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />15<br />Get SSURGO Soil Data & Get EPA <br /><ul><li> Accessing geospatial soil data via Soil Data Access(SDA) web services, currently only for small areas.
    52. 52. Accessing National Hydrography Dataset( watershed and stream shapefile) via EPA Geospatial Services.</li></li></ul><li>Department of Civil, Architectural & Environmental Engineering<br />16<br />2. Data Processing Library<br /><ul><li>Geospatial Data Processing
    53. 53. Delineate watershed/sub-watershed boundary, Generate river network; Create Triangulated Irregular Network(TIN); Process Soil, Land Cover data; Create Hydrologic Response Unit (HRU).
    54. 54. Time Series Processing
    55. 55. Interpolation/Extrapolation, Unit Conversion.
    56. 56. Data processing customized for data sources
    57. 57. Aggregate NLDAS-2, MPE gridded data for sub-watersheds.</li></ul>6/2/2011<br />
    58. 58. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />17<br />DEM Processing<br /><ul><li> Perform DEM processing step by step — based on the procedures deployed in the Terrain Analysis Using Digital Elevation Models (TauDEM)</li></li></ul><li>Department of Civil, Architectural & Environmental Engineering<br />17<br />DEM Processing<br /><ul><li> Perform DEM processing step by step — based on the procedures deployed in the Terrain Analysis Using Digital Elevation Models (TauDEM)
    59. 59. Locate the outlet</li></ul>6/2/2011<br />
    60. 60. Department of Civil, Architectural & Environmental Engineering<br />18<br />Geospatial Data Processing<br /><ul><li> Perform DEM processing and TIN generation via WPS web services </li></ul>Client <br />Server <br />Watershed Delineation<br /><ul><li>DEM Processing</li></ul> - Delineate watershed boundary<br /> - Generate river system<br /> - Divide subbasins<br /><ul><li> TIN Generation</li></ul> -Delaunay Triangulation<br />WPS web services <br />Watershed Triangulation<br /><ul><li>WPS(OpenGIS ® Web Processing Service) provides rules to standardize inputs and outputs for geospatial processing services, and to request execution of a process and handle output from the process.</li></ul>6/2/2011<br />
    61. 61. Department of Civil, Architectural & Environmental Engineering<br />18<br />Geospatial Data Processing<br /><ul><li> Perform DEM processing and TIN generation via WPS web services </li></ul>Client <br />Server <br /><ul><li> via Local activities</li></ul>Watershed Delineation<br /><ul><li>DEM Processing</li></ul> - Delineate watershed boundary<br /> - Generate river system<br /> - Divide subbasins<br /><ul><li> TIN Generation</li></ul> -Delaunay Triangulation<br />WPS web services <br />Watershed Triangulation<br /><ul><li>WPS(OpenGIS ® Web Processing Service) provides rules to standardize inputs and outputs for geospatial processing services, and to request execution of a process and handle output from the process.</li></ul>6/2/2011<br />
    62. 62. Department of Civil, Architectural & Environmental Engineering<br />19<br />Creating Hydrologic Response Unit<br />Step 1: Processing SSURGO Soil Data<br /><ul><li> For soil data accessed via SDA web service</li></ul>(1) Merge map units based on hydro groups<br />(2) Clip the shapefile with watershed boundary<br /><ul><li> For soil data accessed via Soil Data Mart</li></ul>(1) Merge map units for each county-based shapefile<br />(2) Merge county-based shapefiles<br />(3) Clip the merged shapefile with watershed boundary<br /><ul><li> Simplify soil groups to A, B, C, D (Optional)</li></ul>6/2/2011<br />
    63. 63. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />20<br />Creating Hydrologic Response Unit<br />Step 2: Processing Land Cover Data<br />Step 3: Create HRU<br />
    64. 64. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />21<br />Processing Time Series Data<br />
    65. 65. Department of Civil, Architectural & Environmental Engineering<br />22<br />Processing NLDAS-2, MPE Gridded data<br /><ul><li> Aggregate Gridded Data for (sub-)Watersheds
    66. 66. For NLDAS-2 gridded data
    67. 67. For MPE gridded data</li></ul>6/2/2011<br />
    68. 68. Department of Civil, Architectural & Environmental Engineering<br />22<br />Processing NLDAS-2, MPE Gridded data<br /><ul><li> Aggregate Gridded Data for (sub-)Watersheds
    69. 69. For NLDAS-2 gridded data
    70. 70. For MPE gridded data</li></ul>6/2/2011<br />
    71. 71. Department of Civil, Architectural & Environmental Engineering<br />23<br />3. Hydrologic Model Library – TOPMODEL<br /><ul><li>TOPMODEL
    72. 72. A physically based, semi-distributed watershed model that simulates hydrologic fluxes.
    73. 73. The VB version converted from 9502 FORTRAN version is migrated into the following workflow. </li></ul>[Activity 1] — Compute Topographic Index Histogram for the whole watershed or each sub-basin.<br />[Activity 2] — Compute Area-Distance Histogram for routing flow.<br />[Activity 3] — Interactive activity for inputting/modifying initial condition and parameters.<br />[Activity 4] — TOPMODEL computation kernel.<br />6/2/2011<br />
    74. 74. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />24<br />3. Hydrologic Model Library – SWAT<br /><ul><li> The hydrology component in SWAT is based on the water balance equation in the soil profile and simulates processes including canopy interception, snow melt, infiltration, surface runoff, evapotranspiration, lateral flow and percolation.
    75. 75. Source codes: SWAT 2005 version</li></li></ul><li>6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />25<br />3. Hydrologic Model Library– Single hydrologic processes<br /><ul><li> Individual Hydrologic Methods</li></ul>Main Stream<br />
    76. 76. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />26<br />4. Post-Analysis & Utilities Library<br /><ul><li>Comparison of simulated and observed hydrographs
    77. 77. Display both hydrographs</li></ul>-- Hydrograph shape<br />-- Peak time and amount<br />-- Time lag or shifts<br /><ul><li> Compute seven performance measures</li></li></ul><li>6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />27<br />4. Post-Analysis & Utilities Library<br /><ul><li>Water Balance Check</li></ul>--Total Amount<br />--Distribution among hydrologic components<br />
    78. 78. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />28<br />Code Implementation<br /><ul><li> Activities
    79. 79. Encoded in C# and compiled into Dynamic Link Libraries (DLL).
    80. 80. Define input/output variables explicitly via a metadata-tagging approach.
    81. 81. Define “Execute” function that is invoked by the engine at run time.
    82. 82. Programming work
    83. 83. Scripting from the ground up.
    84. 84. Converting legacy codes from other language to C#.</li></ul> e.g. from VB to C# (TOPMODEL)<br /> from C to C# (SWAT)<br /><ul><li> Compiling legacy codes into DLL, </li></ul> and creating invoker interface to <br /> access functions within the DLL. <br />
    85. 85. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />29<br />Data Structure<br /><ul><li> Major components
    86. 86. physical computation elements: Sub-basin, River, HRU
    87. 87. Time Series: gridded or gauge-based. </li></li></ul><li>30<br />Applications<br /><ul><li> Schuylkill Watershed: Located in Southeastern Pennsylvania, U.S.A. The river is approximately 209km in length, and the watershed covers about 5,229 sq.km. </li></ul>Department of Civil, Architectural & Environmental Engineering<br /><ul><li> Studies
    88. 88. apply the SWAT workflow to simulate daily runoff hydrographs over a 4 year period ranging from 2005 to 2008.
    89. 89. apply the TOPMODEL workflow along with a loosely coupled hydrologic model for the simulation of a flood event.
    90. 90. Sub-studies
    91. 91. conduct DEM processing via three types of workflows, and subdivide watershed under two schemes.
    92. 92. analyze precipitation data accessed from different data sources.
    93. 93. estimate potential evapotranpiration using activities encapsulating different approaches. </li></ul>6/2/2011<br />
    94. 94. Department of Civil, Architectural & Environmental Engineering<br />31<br />Step 1: Delineate watershed and generate river network<br /><ul><li> Schuylkill DEM: Cell size 1 arc second,4950*3826 cells, Geospatial Extent (39.86,-76.4,40.9,-75.1)
    95. 95. Workflows: 1)Step by Step workflow, 2)Terrain Processing workflow, 3)Web service based workflow
    96. 96. Delineation: 1) 7 sub-basins: 500,000 cells as threshold </li></ul> 2) 33 sub-basins: 100,000 cells as threshold<br />Total Flow Path<br />Sink Filled DEM<br />Flow Direction<br />Flow Accumulation<br />Raw DEM<br />Watershed Grid <br />Stream order<br /> Watershed and River Network (.shp)<br /> Stream Raster<br />6/2/2011<br />
    97. 97. Department of Civil, Architectural & Environmental Engineering<br />32<br />Step 2: Create Hydrologic Response Units<br /><ul><li>Reclassified Land Cover Data
    98. 98. Merged Soil Groups
    99. 99. HRUS</li></ul>-- 7 sub-basin watershed contains 23 HRUs <br />-- 33 sub-basin watershed contains 114 HRUs <br />6/2/2011<br />
    100. 100. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />33<br />Step 3: Access and process meteorological data <br /><ul><li>Get precipitation data from NLDAS-2 and NWS MPE
    101. 101. Temperature, Solar radiation, wind speed, pressure are accessed from NLDAS-2</li></li></ul><li>Department of Civil, Architectural & Environmental Engineering<br />33<br />Step 3: Access and process meteorological data <br /><ul><li>Get precipitation data from NLDAS-2 and NWS MPE
    102. 102. The NLDAS and NWIS precipitation data exhibit a good correlation.
    103. 103. The NLDAS precipitation data are adopted in the following modeling.
    104. 104. Temperature, Solar radiation, wind speed, pressure are accessed from NLDAS-2</li></ul>6/2/2011<br />
    105. 105. Department of Civil, Architectural & Environmental Engineering<br />34<br />Step 4: Compute Daily Potential Evapotranspiration<br /><ul><li>PET Approaches
    106. 106. Penman-Monteith</li></ul>----temperature, atmospheric pressure, relative humidity, solar radiation, wind speed<br /><ul><li> Priestley-Taylor</li></ul>----temperature, atmospheric pressure, relative humidity, solar radiation<br /><ul><li> Hargreaves </li></ul>----temperature<br /><ul><li>Thornthwaite</li></ul>----temperature<br /><ul><li> The Thornthwaite underestimates the PET significantly, while the other three methods exhibit close simulations.
    107. 107. The estimates of Penman-Monteith method are adopted in the following modeling. </li></ul>6/2/2011<br />
    108. 108. Department of Civil, Architectural & Environmental Engineering<br />35<br />Step 5: Simulating Daily Hydrographs via SWAT workflow<br /><ul><li>Workflow: run for 7-sub-basin and 33-sub-basin watershed, over a period of 4 years(2005-2008)</li></ul>----not fit well. <br />----no significant difference between two simulations. <br />6/2/2011<br />
    109. 109. Department of Civil, Architectural & Environmental Engineering<br />36<br />Step 5: Simulating Daily Hydrographs via SWAT workflow<br /><ul><li>Water Balance Check</li></ul>Difference=Precipitation-Evaporation-Runoff-∆Soil Water<br /><ul><li>Water Balance Comparison </li></ul> over Sub-basins<br /><ul><li> Add river discharge as an additional input in computing water balance for each sub-basin.
    110. 110. Aggregate the water balance of finer sub-basins belonging to each coarse sub-basin.
    111. 111. The water balance accumulated from that of finer sub-basins is close to the one of corresponding coarse sub-basin. </li></ul>6/2/2011<br />
    112. 112. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />37<br />Step 6: Simulating hourly hydrographs via TOPMODEL workflow & a loosely coupled hydrologic model workflow<br /><ul><li>TOPMODEL workflow
    113. 113. Flood Event</li></ul>----October 8th -11th ,2005<br /><ul><li> Inputs</li></ul>----NLDAS-2 hourly precipitation<br />----Penman-Monteith PET<br /><ul><li>Loosely coupled hydrologic model workflow
    114. 114. Inputs</li></ul>----NLDAS-2 hourly precipitation<br />
    115. 115. 6/2/2011<br />Department of Civil, Architectural & Environmental Engineering<br />37<br />Step 6: Simulating hourly hydrographs via TOPMODEL workflow & a loosely coupled hydrologic model workflow<br /><ul><li>TOPMODEL workflow
    116. 116. Flood Event</li></ul>----October 8th -11th ,2005<br /><ul><li> Inputs</li></ul>----NLDAS-2 hourly precipitation<br />----Penman-Monteith PET<br /><ul><li>Loosely coupled hydrologic model workflow
    117. 117. Inputs</li></ul>----NLDAS-2 hourly precipitation<br />
    118. 118. Department of Civil, Architectural & Environmental Engineering<br />38<br />Discussions<br /><ul><li>Applicability of HCMS
    119. 119. In general, it is remarkably straightforward to build up workflows in the HCMS for hydrologic modeling purposes.
    120. 120. It can save time and effort through the automated execution that the workflow sequences afford.
    121. 121. With the nationwide data coverage of incorporated data sources, the HCMS can be applied to anywhere in the US.
    122. 122. Performance of HCMS
    123. 123. Execution time losses </li></ul>---- The workflow engine needs 15-40 seconds to initiate the workflow before execution. <br />---- The interactive activity takes an additional 5-15 seconds to start up the interactive window.<br />The losses are insignificant for workflows involving heavy computations, e.g. DEM processing for large areas; However, they are indeed considered as a burden for the workflows need less run time. <br /><ul><li> Saved time</li></ul>---- in the preparation of model input.<br />---- in parallel execution using web service based activities.<br />6/2/2011<br />
    124. 124. Department of Civil, Architectural & Environmental Engineering<br />39<br />Summary<br /><ul><li>The purpose of our work is to design a hydrologic community modeling system(HCMS) that permits the seamless integration of data flows from source, to preparation, to ingestion, to model execution, to harvesting and analysis of result data.
    125. 125. TRIDENT workflow system provides a platform for designing the HCMS and for assembling hydrologic models as workflow sequences.
    126. 126. The HCMS was tested by carrying out several typical hydrologic modeling studies over Schuylkill watershed. It is proved to be used quite well as a modeling platform. While it is not computational cost free due to the middle ware layer, the additional time consumption is “affordable”, especially in the lengthy data preparation arena. </li></ul>6/2/2011<br />
    127. 127. Department of Civil, Architectural & Environmental Engineering<br />40<br />Future Work<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    128. 128. Department of Civil, Architectural & Environmental Engineering<br />40<br />Future Work<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    129. 129. Department of Civil, Architectural & Environmental Engineering<br />40<br />Future Work<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Data<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Model<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />Tool<br />6/2/2011<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×