Process Automation forHydrological Data Mappingover GIS SoftwareBy Rohan Jain (08AG1016)
Introduction● Weather data is available from various organisations like IMD, CWC through their stations spanning all over the country, periodically.● The data available from these places can be used for further processing.● Processing is done via various GIS Software available.● ArcGIS is one such popular software. It is used for this project
Introduction: Problem● Data is not available in format ArcGIS support● So it cannot be directly imported● Manually importing 10s of thousands of data is not possible.● Hence data needs to be automatically converted into an ArcGIS format.● But again data from all the sources is not in a standardised format.● So each data source needs special attention
Objectives● Automatic conversion of existing hydrological data of Mahanadi river basin into a universal time-series format● Mapping of the data into ArcHydro model of the ArcGIS software
Study Area: Description● Mahandi river basin, located between longitudes 800 30 and 870 E, and latitudes 190 21 and 230 35 N● 4.3% of the total geographical area of India● Mahanadi was notorious for its devastating floods.● Hirakud Dam, one of the longest dams improved the situation greatly.
Study Area: Data Available● Data from India Meteorological Department and Central Water Commission (CWC)● Rainfall data● Escape Discharge data● Water Level Data● Data from remote sensing
Methodology: Study Material● Book: ArcHydro - GIS for Water Resources by David R. Maidment● Book: Arc Hydro Tools - Tutorials● GIS Course Content - University of Texas● Web Resources, Lectures made available by ESRI (ArcGIS Developer organisation)
Methodology● For interfacing with ArcGIS dBase (*.dbf) database file format used● dBase is a popular database and ArcGIS relies on it itself for storing data, so a good choice for using it for our task● Python libraries available (dbfpy)● For data model to store the time series, used the TimeSeries model from ArcHydro data models.
Methodology: Data Model● FeatureID: ID of the feature for which this time series data exists. IMD Stations, CWC Gauges etc.● TSTypeID: ID of the time series type. We have Precipitaion, Discharge, Water Level etc defined● TSDateTime: The date and time of individual data● TSValue: Individual data value
Methodology: Automation1. The data obtained from various organisations is converted into a format which follows python data structures.2. Separate (dBase) files contain information about HydroIDs (which will help find FeatureID). The information is extracted and used to find FeatureIDs for station names3. Time Series is generated and then further published as dBase files for use with ArcGIS software.
The Data ConversionProcess
Methodology: Code Written● Modules ○ These are for generic tasks which are applicable to all data sources ○ timeseries.py ■ Takes care of timeseries related internal tasks ■ Also generates the dBase files ○ stations.py: ■ Process the HydroIDs (FeatureIDs in Time Series database) ■ Fetches ID - Name info about the stations
Methodology: Code Written● Individual Data Source Scripts ○ Since each data source provides information in a different format, they all need a separate script. ○ These scripts process the raw data to pythonic format and then generate time series database● Written in Python Programming Language● Total roughly 450 lines of python code● A C/Java equivalent will easily measure 2-3 times
Results● Set up an initial project with correct directory hierarchy and install python + the required libraries● Then, on execution of the scripts the time series files are generated automatically● The time series files can then be imported into ArcGIS table
Results: Loading DataLoading datainto a TimeSeries table inArcCatalog
Result: Loading DataArcCatalog data loading dialogs
Result: Loading Data Displaying data after being imported.
Result: Processing Data Processing the data in ArcMap using ArcHydro tools
Result: Processing DataArcMap Processing the Discharge Time Series
Future Work● Rewrite the modules using Object Oriented Approach to improve the code quality and future additions of code easier● Apart from this Rainfall, Discharge, Water Level series more data can be obtained and added