This document provides guidance for managing hydro-meteorological data in India within a Hydrological Information System (HIS). It discusses the data lifecycle, from monitoring networks and data collection to analysis, dissemination and use. It directs the user to relevant manuals on topics like rainfall, snow, climate and evaporation data processing. The goal is to standardize procedures and provide high quality data to inform water resources planning and management.
This document provides an overview of guidance materials for the management of surface water data within India's Hydrological Information System (HIS). It describes the lifecycle of hydrometric data from collection through analysis and publication. Key documents that provide procedures for surface water data management are the HIS Manual Surface Water and various training modules developed under the Hydrology Project. The manual and modules cover topics like network design, data collection, entry, validation, processing, analysis, and dissemination of water level, stage-discharge, and flow data. The goal is to standardize surface water data management practices across states and agencies to improve data quality and usability.
This document provides guidance for managing groundwater data as part of a Hydrological Information System (HIS) in India. It discusses the lifecycle of hydrometric data from collection to dissemination. Key points covered include:
- The HIS Manual Groundwater is the primary reference for groundwater data management procedures.
- Groundwater data goes through stages of monitoring, data sensing, validation, analysis, and publication.
- Sections provide guidance on groundwater monitoring networks, data collection, processing, analysis, and dissemination of data.
- Tables list the relevant HIS Manual volumes for groundwater level and rainfall data management.
This document provides a readers' guide to groundwater documents produced by the Hydrology Project (HP). It summarizes the Hydrological Information System (HIS) Manual Groundwater, which consists of 10 volumes covering topics like hydro-meteorology, geo-hydrology, water quality sampling and analysis, data processing, and more. It also describes related groundwater training modules and other documents on standards, maintenance, and more. The guide is intended to help users understand and locate relevant groundwater information resources in the HIS document library.
This document provides a readers' guide to surface water documents related to India's Hydrological Information System (HIS). It summarizes the key documents, including the 10-volume HIS Manual Surface Water, which describes procedures for surface water data collection, analysis and management. It also outlines the surface water training modules available, which cover topics like hydrometry, meteorology and data processing. The guide is intended to help users understand and locate relevant surface water documents on the Hydrology Project website.
This document provides a readers' guide to hydro-meteorological documents related to India's Hydrological Information System (HIS). It summarizes key hydro-meteorology manuals, training modules, and other documents produced by the Hydrology Project to support the collection, processing, and use of rainfall and climate data in India. The primary references are the HIS Manual Surface Water and Groundwater, which describe procedures for network design, data collection, processing, and analysis of hydro-meteorological data. Related training modules cover topics like rainfall data entry, validation, analysis and reporting. The guide aims to help users locate relevant hydro-meteorology documents.
This document provides a guide to water quality documents produced by the Hydrology Project in India. It summarizes the key water quality documents, including the HIS Manual Water Quality, Water Quality Training Modules, and additional technical papers. The guide is intended to help users locate relevant water quality information for surface water and groundwater monitoring and analysis.
This document provides an overview of a Hydrological Information System (HIS) being developed for 9 states in India. It discusses the key components and activities of the HIS, which include: assessing user needs, establishing observation networks, managing historical data, collecting field data, processing and analyzing data, exchanging and reporting data, storing and disseminating data, and developing institutional and human resources. The overall goal of the HIS is to provide reliable hydrological data and information to support long-term water resources planning and management decisions in India.
The document provides guidance on assessing hydrological data needs through stakeholder interviews. Small interview teams will visit existing and potential hydrological data users with a questionnaire. The questionnaire aims to gather information on: 1) The user's organizational profile, current water system use, and current data availability and sources. 2) The user's future hydrological data classification, proposed uses, and parameter requirements. Interview teams will explain the questionnaire and hydrological information system, then review responses to ensure questions are understood and data needs are properly assessed. Results will inform immediate data provision and long-term system adjustments.
This document provides an overview of guidance materials for the management of surface water data within India's Hydrological Information System (HIS). It describes the lifecycle of hydrometric data from collection through analysis and publication. Key documents that provide procedures for surface water data management are the HIS Manual Surface Water and various training modules developed under the Hydrology Project. The manual and modules cover topics like network design, data collection, entry, validation, processing, analysis, and dissemination of water level, stage-discharge, and flow data. The goal is to standardize surface water data management practices across states and agencies to improve data quality and usability.
This document provides guidance for managing groundwater data as part of a Hydrological Information System (HIS) in India. It discusses the lifecycle of hydrometric data from collection to dissemination. Key points covered include:
- The HIS Manual Groundwater is the primary reference for groundwater data management procedures.
- Groundwater data goes through stages of monitoring, data sensing, validation, analysis, and publication.
- Sections provide guidance on groundwater monitoring networks, data collection, processing, analysis, and dissemination of data.
- Tables list the relevant HIS Manual volumes for groundwater level and rainfall data management.
This document provides a readers' guide to groundwater documents produced by the Hydrology Project (HP). It summarizes the Hydrological Information System (HIS) Manual Groundwater, which consists of 10 volumes covering topics like hydro-meteorology, geo-hydrology, water quality sampling and analysis, data processing, and more. It also describes related groundwater training modules and other documents on standards, maintenance, and more. The guide is intended to help users understand and locate relevant groundwater information resources in the HIS document library.
This document provides a readers' guide to surface water documents related to India's Hydrological Information System (HIS). It summarizes the key documents, including the 10-volume HIS Manual Surface Water, which describes procedures for surface water data collection, analysis and management. It also outlines the surface water training modules available, which cover topics like hydrometry, meteorology and data processing. The guide is intended to help users understand and locate relevant surface water documents on the Hydrology Project website.
This document provides a readers' guide to hydro-meteorological documents related to India's Hydrological Information System (HIS). It summarizes key hydro-meteorology manuals, training modules, and other documents produced by the Hydrology Project to support the collection, processing, and use of rainfall and climate data in India. The primary references are the HIS Manual Surface Water and Groundwater, which describe procedures for network design, data collection, processing, and analysis of hydro-meteorological data. Related training modules cover topics like rainfall data entry, validation, analysis and reporting. The guide aims to help users locate relevant hydro-meteorology documents.
This document provides a guide to water quality documents produced by the Hydrology Project in India. It summarizes the key water quality documents, including the HIS Manual Water Quality, Water Quality Training Modules, and additional technical papers. The guide is intended to help users locate relevant water quality information for surface water and groundwater monitoring and analysis.
This document provides an overview of a Hydrological Information System (HIS) being developed for 9 states in India. It discusses the key components and activities of the HIS, which include: assessing user needs, establishing observation networks, managing historical data, collecting field data, processing and analyzing data, exchanging and reporting data, storing and disseminating data, and developing institutional and human resources. The overall goal of the HIS is to provide reliable hydrological data and information to support long-term water resources planning and management decisions in India.
The document provides guidance on assessing hydrological data needs through stakeholder interviews. Small interview teams will visit existing and potential hydrological data users with a questionnaire. The questionnaire aims to gather information on: 1) The user's organizational profile, current water system use, and current data availability and sources. 2) The user's future hydrological data classification, proposed uses, and parameter requirements. Interview teams will explain the questionnaire and hydrological information system, then review responses to ensure questions are understood and data needs are properly assessed. Results will inform immediate data provision and long-term system adjustments.
This document provides guidance on sampling principles for hydrological and hydro-meteorological variables. It discusses key concepts such as units of measurement, basic statistics, measurement error, sampling frequency and spatial sampling. The goal is to design monitoring networks that can estimate important statistical parameters about variables while accounting for various sources of error from sampling. Basic statistical concepts covered include distribution functions, parameters like mean and variance, and how to estimate these from samples along with associated confidence intervals and effects of serial correlation.
This document describes procedures for surface water data processing under the Hydrological Information System (HIS) in India. It discusses various stages of data processing including receipt of data, data entry, validation, completion, compilation, analysis, reporting and transfer. It emphasizes the importance of validation to correct errors and identify data reliability. Validation is done at multiple levels - primary, secondary and hydrological. The document also covers organizing temporary databases, transferring data between databases, and backing up databases.
This document outlines the stages of surface water data processing under the Hydrological Information System (HIS) in India. It discusses: 1) Receipt of data from field stations and storage of raw records; 2) Data entry at sub-divisional offices; 3) Validation of data through primary, secondary, and hydrological checks; 4) Completion and correction of missing or erroneous data; 5) Compilation, analysis, and reporting of validated data; 6) Transfer of data between processing levels from sub-division to division to state centers. The overall goal is to process field data in a systematic series of steps to produce quality-controlled hydrological information.
Mh sw optimisation of g&d stations network of maharashtrahydrologyproject0
This document discusses optimizing the streamgauge and raingauge network for the Upper Bhima Basin in India. It provides background on hydrological information systems and networks in India. The Hydrology Project aims to improve India's capabilities for collecting and analyzing hydrological data. This study was conducted as part of the Hydrology Project to review and optimize the existing hydrometric network in Maharashtra state, which includes streamgauges and raingauges. The goal is to ensure the network is collecting the necessary data to facilitate optimal water resources use and management in the Upper Bhima Basin.
This document describes the design of a hydrological data storage and dissemination system. It discusses the major components of the system including databases to store different types of hydrological data (e.g. field data, processed data, maps), a catalogue to allow users to search and access data, and interfaces to allow external organizations and users to input and retrieve data. It provides specifications for hardware, software, security, and other technical aspects required to build the hydrological information system. The overall aim is to create a centralized, standardized system for permanently storing all types of hydrological data from various agencies and making it accessible to authorized users.
This document provides guidance on using regression analysis for data validation in hydrological data processing. It discusses simple linear regression, multiple linear regression, and stepwise regression. Regression analysis can be used to validate and fill in missing water level, rainfall, and discharge data. It establishes relationships between dependent and independent variables. Both linear and nonlinear regression models are used in hydrological applications. Key applications mentioned include rating curves, spatial interpolation of rainfall, and validating station data against nearby stations.
This document provides guidance on how to carry out primary validation of water level data. It discusses validating data from staff gauges, automatic water level recorders, and digital water level recorders by checking for errors and inconsistencies in single time series, and by comparing data between instruments. Methods include examining data graphically and against physical limits, and viewing hydrographs from adjacent stations. The goal is to flag potentially incorrect values for further validation while replacing others with corrected values based on these initial checks.
This document provides guidance on network design and site selection for hydro-meteorological stations. It discusses the steps for network optimization which include reviewing existing networks, identifying data needs, prioritizing objectives, determining required network density, and cost estimation. Site selection considerations are also outlined, including technical, environmental, logistical, security, legal and financial aspects. Key factors for siting stations include exposure conditions, wind protection, level ground, and integrating with other monitoring networks.
Achievement of new state under hp 2 - himachal pradesh in integrated water ma...hydrologywebsite1
This document provides an overview of the Hydrology Project Phase II being implemented in Himachal Pradesh. Some key points:
1. The project aims to improve hydrological data collection and management in HP to support water resource planning. It has three main components - institutional strengthening, network expansion, and recurrent costs.
2. Activities under the project include expanding the network of rain gauges, piezometers, weather stations; establishing state and divisional data centers; upgrading water quality labs; and conducting purpose-driven studies.
3. Over 650 officials have been trained so far. MoUs have also been signed for data sharing with other agencies. Workshops and study tours have been held to raise awareness about
The document provides information about a workshop on standards for groundwater monitoring, processing, and data dissemination. It includes the following key points:
1. The workshop aims to review current practices and adopt standard formats, techniques, and procedures for computerized groundwater data acquisition, processing, validation, retrieval and dissemination.
2. Topics to be addressed include computerized techniques, data standards, quality monitoring objectives and procedures, dedicated software demonstrations, and requirements for software.
3. The 3-day workshop program includes sessions on data standards, software discussions, and a visit to an operational digital monitoring network site. Standardizing procedures and using computerization can help establish a reliable hydrological information system.
This document provides guidance on reporting climatic data in India. It discusses the purpose and contents of annual reports on climatic data, including evaporation data. Key points covered include:
- Annual reports summarize evaporation data for the reporting year and compare to long-term statistics.
- Reports include details on the observational network, basic evaporation statistics, data validation processes.
- Network maps and station listings provide details on locations and recorded variables. Statistics include monthly and annual summaries for the current year and historical averages.
- Reports aim to inform users and support planning, while also recognizing data producers and maintaining the climatic observation system.
The Hydrology Project established India's Hydrological Information System by developing networks of hydro-meteorological stations, web-based data management systems, and tools for water resources planning and management. It involved 29 agencies across 13 states and 8 central government organizations. Key achievements include establishing surface and groundwater observation networks, databases for water quality and quantity data, decision support systems for integrated planning, and capacity building for water resource professionals. The project helped shift from isolated development to comprehensive basin-scale planning and management of water resources.
The document summarizes efforts to upgrade India's hydrological information system through the Hydrology Project. Key aspects of the upgrade include standardizing data collection procedures, developing infrastructure like new observation stations, and establishing a comprehensive computerized database. Over 1,700 existing rainfall stations were reactivated or upgraded, 650 new river gauging stations were established, and 7,900 new groundwater observation wells were added. The upgraded system aims to provide reliable, accessible hydrological data to support improved water management across nine states in India covering 1.7 million square kilometers.
This document provides guidance on entering rainfall data into a dedicated hydrological data processing software (SWDES). It discusses entering daily rainfall data, twice daily rainfall data, and hourly rainfall data from manual records or digital loggers. The key steps described are manually inspecting field records before entry, using customized SWDES forms that match field sheets, entering values and computing totals, plotting time series graphs, and performing data validation checks. The overall aim is to efficiently digitize rainfall observations near monitoring stations for further validation and analysis.
This document provides guidance on managing groundwater data within India's Hydrological Information System (HIS). It discusses the lifecycle of hydrometric data from collection to dissemination. The document directs the user to relevant manuals within HIS, particularly the Groundwater manual, for guidance on groundwater level monitoring networks, data collection, processing, analysis and publication. It describes the various types of manuals within HIS - design, field operation, and reference - and lists the specific volumes and parts most pertinent to groundwater level data. The overall aim is to help users locate and understand documentation to standardize high quality groundwater data management and inform water resource planning.
This document provides guidance for managing sediment and water quality data within India's Hydrological Information System (HIS). It summarizes the key HIS manuals that provide procedures for monitoring, data collection, validation, analysis, dissemination and publication of sediment and water quality data. Specifically, it outlines the multi-volume HIS Manuals for Surface Water and Groundwater, which describe the lifecycle of sediment and water quality data within the HIS. It also lists some additional HPI documentation and training modules that are relevant to sediment and water quality monitoring and analysis. The overall aim is to help users navigate and understand the various documents within the HIS library to properly manage sediment and water quality data.
This document describes procedures for surface water data processing under the Hydrological Information System (HIS) in India. It discusses various stages of data processing including receipt of data, data entry, validation, completion, compilation, analysis, reporting and transfer. It emphasizes the importance of validation to correct errors and identify data reliability. Validation is done at multiple levels - primary, secondary and hydrological. The document also covers organizing temporary databases, transferring data between databases, and backing up databases.
This document describes procedures for surface water data processing under the Hydrological Information System (HIS) in India. It discusses various stages of data processing including receipt of data, data entry, validation, completion, compilation, analysis, reporting and transfer. It emphasizes the importance of validation to correct errors and identify data reliability. Validation is carried out at multiple levels, from primary validation during data entry to secondary and hydrological validation. The document also covers organizing temporary databases, transferring data between databases, and backing up databases.
This document provides guidance on reporting climatic data in India. It discusses the purpose and contents of annual reports on climatic data, including evaporation data. Key points covered include:
- Annual reports summarize evaporation data for the reporting year and compare to long-term statistics.
- Reports include details on the observational network, basic evaporation statistics, data validation processes.
- Network maps and station listings provide details of monitoring locations. Statistics include monthly and annual evaporation amounts for the current year and historical averages.
- Reports aim to inform water resource planning, acknowledge data collection efforts, and provide access to climatic data records.
This document provides guidance on using regression analysis for data validation in hydrological data processing. It discusses simple linear regression, multiple linear regression, and stepwise regression. Regression analysis can be used to validate and fill in missing water level, rainfall, and discharge data. It establishes relationships between dependent and independent variables. Both linear and nonlinear regression models are used in hydrological applications. Key applications mentioned include rating curves, spatial interpolation of rainfall, and validating station data against nearby stations.
This document provides guidance on using regression analysis for data validation in hydrological data processing. It discusses simple linear regression, multiple linear regression, and stepwise regression. Regression analysis can be used to validate and fill in missing water level, rainfall, and discharge data. It establishes relationships between dependent and independent variables. Both linear and nonlinear regression models are used in hydrological applications. Key applications mentioned include rating curves, spatial interpolation of rainfall, and validating station data against nearby stations.
1. The document describes India's Hydrological Information System (HIS) which collects, processes, stores and disseminates hydrological data.
2. The HIS aims to provide reliable data to support long-term water resources planning and management by establishing observation networks, managing historical data, collecting and processing data, and storing and disseminating it to users.
3. Key activities of the HIS include assessing user needs, establishing and maintaining observation networks, managing historical data, collecting field data, processing and analyzing data, exchanging and reporting data, and storing and disseminating data to support water planning in India.
This document provides guidance on sampling principles for hydrological and hydro-meteorological variables. It discusses key concepts such as units of measurement, basic statistics, measurement error, sampling frequency and spatial sampling. The goal is to design monitoring networks that can estimate important statistical parameters about variables while accounting for various sources of error from sampling. Basic statistical concepts covered include distribution functions, parameters like mean and variance, and how to estimate these from samples along with associated confidence intervals and effects of serial correlation.
This document describes procedures for surface water data processing under the Hydrological Information System (HIS) in India. It discusses various stages of data processing including receipt of data, data entry, validation, completion, compilation, analysis, reporting and transfer. It emphasizes the importance of validation to correct errors and identify data reliability. Validation is done at multiple levels - primary, secondary and hydrological. The document also covers organizing temporary databases, transferring data between databases, and backing up databases.
This document outlines the stages of surface water data processing under the Hydrological Information System (HIS) in India. It discusses: 1) Receipt of data from field stations and storage of raw records; 2) Data entry at sub-divisional offices; 3) Validation of data through primary, secondary, and hydrological checks; 4) Completion and correction of missing or erroneous data; 5) Compilation, analysis, and reporting of validated data; 6) Transfer of data between processing levels from sub-division to division to state centers. The overall goal is to process field data in a systematic series of steps to produce quality-controlled hydrological information.
Mh sw optimisation of g&d stations network of maharashtrahydrologyproject0
This document discusses optimizing the streamgauge and raingauge network for the Upper Bhima Basin in India. It provides background on hydrological information systems and networks in India. The Hydrology Project aims to improve India's capabilities for collecting and analyzing hydrological data. This study was conducted as part of the Hydrology Project to review and optimize the existing hydrometric network in Maharashtra state, which includes streamgauges and raingauges. The goal is to ensure the network is collecting the necessary data to facilitate optimal water resources use and management in the Upper Bhima Basin.
This document describes the design of a hydrological data storage and dissemination system. It discusses the major components of the system including databases to store different types of hydrological data (e.g. field data, processed data, maps), a catalogue to allow users to search and access data, and interfaces to allow external organizations and users to input and retrieve data. It provides specifications for hardware, software, security, and other technical aspects required to build the hydrological information system. The overall aim is to create a centralized, standardized system for permanently storing all types of hydrological data from various agencies and making it accessible to authorized users.
This document provides guidance on using regression analysis for data validation in hydrological data processing. It discusses simple linear regression, multiple linear regression, and stepwise regression. Regression analysis can be used to validate and fill in missing water level, rainfall, and discharge data. It establishes relationships between dependent and independent variables. Both linear and nonlinear regression models are used in hydrological applications. Key applications mentioned include rating curves, spatial interpolation of rainfall, and validating station data against nearby stations.
This document provides guidance on how to carry out primary validation of water level data. It discusses validating data from staff gauges, automatic water level recorders, and digital water level recorders by checking for errors and inconsistencies in single time series, and by comparing data between instruments. Methods include examining data graphically and against physical limits, and viewing hydrographs from adjacent stations. The goal is to flag potentially incorrect values for further validation while replacing others with corrected values based on these initial checks.
This document provides guidance on network design and site selection for hydro-meteorological stations. It discusses the steps for network optimization which include reviewing existing networks, identifying data needs, prioritizing objectives, determining required network density, and cost estimation. Site selection considerations are also outlined, including technical, environmental, logistical, security, legal and financial aspects. Key factors for siting stations include exposure conditions, wind protection, level ground, and integrating with other monitoring networks.
Achievement of new state under hp 2 - himachal pradesh in integrated water ma...hydrologywebsite1
This document provides an overview of the Hydrology Project Phase II being implemented in Himachal Pradesh. Some key points:
1. The project aims to improve hydrological data collection and management in HP to support water resource planning. It has three main components - institutional strengthening, network expansion, and recurrent costs.
2. Activities under the project include expanding the network of rain gauges, piezometers, weather stations; establishing state and divisional data centers; upgrading water quality labs; and conducting purpose-driven studies.
3. Over 650 officials have been trained so far. MoUs have also been signed for data sharing with other agencies. Workshops and study tours have been held to raise awareness about
The document provides information about a workshop on standards for groundwater monitoring, processing, and data dissemination. It includes the following key points:
1. The workshop aims to review current practices and adopt standard formats, techniques, and procedures for computerized groundwater data acquisition, processing, validation, retrieval and dissemination.
2. Topics to be addressed include computerized techniques, data standards, quality monitoring objectives and procedures, dedicated software demonstrations, and requirements for software.
3. The 3-day workshop program includes sessions on data standards, software discussions, and a visit to an operational digital monitoring network site. Standardizing procedures and using computerization can help establish a reliable hydrological information system.
This document provides guidance on reporting climatic data in India. It discusses the purpose and contents of annual reports on climatic data, including evaporation data. Key points covered include:
- Annual reports summarize evaporation data for the reporting year and compare to long-term statistics.
- Reports include details on the observational network, basic evaporation statistics, data validation processes.
- Network maps and station listings provide details on locations and recorded variables. Statistics include monthly and annual summaries for the current year and historical averages.
- Reports aim to inform users and support planning, while also recognizing data producers and maintaining the climatic observation system.
The Hydrology Project established India's Hydrological Information System by developing networks of hydro-meteorological stations, web-based data management systems, and tools for water resources planning and management. It involved 29 agencies across 13 states and 8 central government organizations. Key achievements include establishing surface and groundwater observation networks, databases for water quality and quantity data, decision support systems for integrated planning, and capacity building for water resource professionals. The project helped shift from isolated development to comprehensive basin-scale planning and management of water resources.
The document summarizes efforts to upgrade India's hydrological information system through the Hydrology Project. Key aspects of the upgrade include standardizing data collection procedures, developing infrastructure like new observation stations, and establishing a comprehensive computerized database. Over 1,700 existing rainfall stations were reactivated or upgraded, 650 new river gauging stations were established, and 7,900 new groundwater observation wells were added. The upgraded system aims to provide reliable, accessible hydrological data to support improved water management across nine states in India covering 1.7 million square kilometers.
This document provides guidance on entering rainfall data into a dedicated hydrological data processing software (SWDES). It discusses entering daily rainfall data, twice daily rainfall data, and hourly rainfall data from manual records or digital loggers. The key steps described are manually inspecting field records before entry, using customized SWDES forms that match field sheets, entering values and computing totals, plotting time series graphs, and performing data validation checks. The overall aim is to efficiently digitize rainfall observations near monitoring stations for further validation and analysis.
This document provides guidance on managing groundwater data within India's Hydrological Information System (HIS). It discusses the lifecycle of hydrometric data from collection to dissemination. The document directs the user to relevant manuals within HIS, particularly the Groundwater manual, for guidance on groundwater level monitoring networks, data collection, processing, analysis and publication. It describes the various types of manuals within HIS - design, field operation, and reference - and lists the specific volumes and parts most pertinent to groundwater level data. The overall aim is to help users locate and understand documentation to standardize high quality groundwater data management and inform water resource planning.
This document provides guidance for managing sediment and water quality data within India's Hydrological Information System (HIS). It summarizes the key HIS manuals that provide procedures for monitoring, data collection, validation, analysis, dissemination and publication of sediment and water quality data. Specifically, it outlines the multi-volume HIS Manuals for Surface Water and Groundwater, which describe the lifecycle of sediment and water quality data within the HIS. It also lists some additional HPI documentation and training modules that are relevant to sediment and water quality monitoring and analysis. The overall aim is to help users navigate and understand the various documents within the HIS library to properly manage sediment and water quality data.
This document describes procedures for surface water data processing under the Hydrological Information System (HIS) in India. It discusses various stages of data processing including receipt of data, data entry, validation, completion, compilation, analysis, reporting and transfer. It emphasizes the importance of validation to correct errors and identify data reliability. Validation is done at multiple levels - primary, secondary and hydrological. The document also covers organizing temporary databases, transferring data between databases, and backing up databases.
This document describes procedures for surface water data processing under the Hydrological Information System (HIS) in India. It discusses various stages of data processing including receipt of data, data entry, validation, completion, compilation, analysis, reporting and transfer. It emphasizes the importance of validation to correct errors and identify data reliability. Validation is carried out at multiple levels, from primary validation during data entry to secondary and hydrological validation. The document also covers organizing temporary databases, transferring data between databases, and backing up databases.
This document provides guidance on reporting climatic data in India. It discusses the purpose and contents of annual reports on climatic data, including evaporation data. Key points covered include:
- Annual reports summarize evaporation data for the reporting year and compare to long-term statistics.
- Reports include details on the observational network, basic evaporation statistics, data validation processes.
- Network maps and station listings provide details of monitoring locations. Statistics include monthly and annual evaporation amounts for the current year and historical averages.
- Reports aim to inform water resource planning, acknowledge data collection efforts, and provide access to climatic data records.
This document provides guidance on using regression analysis for data validation in hydrological data processing. It discusses simple linear regression, multiple linear regression, and stepwise regression. Regression analysis can be used to validate and fill in missing water level, rainfall, and discharge data. It establishes relationships between dependent and independent variables. Both linear and nonlinear regression models are used in hydrological applications. Key applications mentioned include rating curves, spatial interpolation of rainfall, and validating station data against nearby stations.
This document provides guidance on using regression analysis for data validation in hydrological data processing. It discusses simple linear regression, multiple linear regression, and stepwise regression. Regression analysis can be used to validate and fill in missing water level, rainfall, and discharge data. It establishes relationships between dependent and independent variables. Both linear and nonlinear regression models are used in hydrological applications. Key applications mentioned include rating curves, spatial interpolation of rainfall, and validating station data against nearby stations.
1. The document describes India's Hydrological Information System (HIS) which collects, processes, stores and disseminates hydrological data.
2. The HIS aims to provide reliable data to support long-term water resources planning and management by establishing observation networks, managing historical data, collecting and processing data, and storing and disseminating it to users.
3. Key activities of the HIS include assessing user needs, establishing and maintaining observation networks, managing historical data, collecting field data, processing and analyzing data, exchanging and reporting data, and storing and disseminating data to support water planning in India.
1. The document outlines the National Water Policy of India which establishes the need for a standardized national hydrological information system to collect, process, and disseminate reliable water resources data.
2. Key goals of the policy include maximizing water availability, integrating surface and groundwater management, preserving environmental and ecological balances, and involving stakeholders in water management.
3. The hydrological information system described in the document is intended to provide the hydrological data and analysis needed to inform planning, design, management, and policy decisions around India's water resources in accordance with the National Water Policy.
This document provides guidance on reporting discharge data from hydrological monitoring stations. It outlines the contents and purpose of yearly reports, including descriptive summaries of streamflow patterns, basic statistics for selected stations, and comparisons to long-term averages. Periodic long-term reports every 5-10 years are also recommended to analyze trends over longer time periods. The reports aim to inform water resource planning and make hydrological data more accessible and understandable for users.
This document provides guidance on collecting, entering, and validating hydrological data for storage and use in a water resources information system in India. It discusses the mandatory information needed for spatial (e.g. well locations) and temporal (e.g. water level measurements) data. It also describes proper data collection procedures like using field forms, maintaining a data collection register, and entering data directly from field forms to reduce errors. The document emphasizes validation of data at multiple stages and storing data according to standards to ensure long-term usability and reliability of the hydrological information system.
This document provides information on a training module for understanding hydrological information system (HIS) concepts and setup. It includes an introduction to HIS, why they are needed, how they are set up under the Hydrology Project. It also discusses who the key users of hydrological data are and how computers are used in hydrological data processing. The training module contains session plans, presentations, handouts, and text to educate participants on HIS objectives, components, and how they provide reliable hydrological data to various end users.
This document provides information on setting up a Hydrological Information System (HIS) for India. It includes details on:
1. Defining key concepts of a HIS, including that it is a system to collect, process, and disseminate hydrological data to provide useful information to users.
2. The need for a standardized HIS in India to better plan for water resources given the variability of water patterns and inadequacies of existing systems.
3. The Hydrology Project aims to improve existing HIS across 8 Indian states to provide more reliable hydrological data for planning and management.
This document discusses next steps after the completion of the Hydrology Project (HP) in India. It summarizes the gains from HP, including establishing an integrated hydrological monitoring network across agencies. Lessons learned include the need for clear expectations and benefits, improved management and implementation approaches, and addressing staffing and training issues. The document proposes expanding HP horizontally to other states and consolidating achievements in states already covered. It also suggests expanding vertically to enable real-time water data use, drought management, and an integrated water resources management system. Institutional reforms are recommended to establish river basin organizations for improved water governance.
This document provides guidance on how to report rainfall data in yearly and periodic reports. It outlines the typical contents and structure of annual reports including descriptive summaries of rainfall patterns, comparisons to long-term averages, basic statistics, and descriptions of major storms. Periodic reports produced every 10 years would include long-term statistics updated over the previous decade as well as frequency analysis of rainfall data. The reports aim to inform stakeholders of rainfall patterns and data availability as well as validate and improve the quality of data collection.
This document provides guidance on how to report rainfall data in yearly and periodic reports. It outlines the typical contents and structure of annual reports including descriptive summaries of rainfall patterns, comparisons to long-term averages, basic statistics, and descriptions of major storms. Periodic reports produced every 10 years would include long-term statistics updated over the previous decade as well as frequency analysis of rainfall data. The reports aim to inform stakeholders of rainfall patterns and data availability as well as validate and improve the quality of data collection.
The document provides guidance on reporting stage discharge data from hydrological monitoring stations. It recommends including a table with summary information for each station such as maximum/minimum observed stages and flows, the number of discharge measurements and ratings developed in the current and previous years, and the standard error of ratings. The purpose is to evaluate monitoring efforts and provide information for planning while avoiding reporting all raw data. Stage discharge relationships and time series data should be made available upon request.
This document provides a final report on the Hydrology Project conducted from 2003 in India with technical assistance from organizations in the Netherlands and India. It summarizes the objectives of establishing a comprehensive Hydrological Information System across various agencies, the activities of the technical assistance provided, and achievements of the project. Key points:
- The project aimed to improve institutional capabilities for hydrological data measurement, collection, analysis and dissemination through a distributed hydrological information system.
- Technical assistance provided support in areas such as assessing user needs, establishing observation networks, data collection/processing, institutional development and training.
- A phased implementation approach was used, starting with planning and standardization before implementation and consolidation of the hydrological information
This document provides guidelines for generating spatial datasets for a hydrology project in India. It outlines 13 themes that will be mapped, including land use, soils, geology, geomorphology, administrative boundaries, hydrologic units, settlements, transportation, drainage, and contours. Standards are provided for mapping scale, projection, accuracy, and database organization. Spatial data will be generated through interpretation of satellite imagery and digitization of existing paper maps. Resulting data will be integrated into surface and groundwater data centers in participating states.
This document provides an operation manual for processing and analyzing groundwater data in India. It details the monitoring networks for water levels, water quality, and hydro-meteorology. It also describes how the data is organized in the Hydrological Information System (HIS) and discusses semi-static and dynamic data collected, including climate/rainfall reviews, groundwater level changes, and resource estimations. The setup of the groundwater agency is explained along with its roles in monitoring, research, and management.
The document summarizes a water purification system that includes a reverse osmosis unit, storage tank, and ultrapure water purification unit. The system uses a microprocessor to control purification from a municipal water supply to produce at least 35 liters/hour of water with a resistivity of 18 megohm cm or higher that is at least 99% free of bacteria, ions, organics, and particles. The system is also capable of continuous monitoring and will automatically shut off if the feed water is inadequate or the storage tank reaches capacity.
The document describes a water purifier using ion exchange resin columns that produces reagent-grade water for trace analysis. It has separate cation and anion exchange columns with a capacity of 25 liters each and can purify water at a rate over 1 liter per minute. Accessories include spare columns, instructions, a cover, and vacuum hose. Key features are an inline conductivity monitor and the ability to regenerate the resin columns.
This document provides specifications for a double distillation water purification unit. It distills water to a conductivity of 1.0 μS/cm or less at 25°C for generating reagent-grade water. The unit is made of quartz glass with a 1.5 liter/hour capacity. It runs on 220V power and includes a metallic stand, ring clamp, and operation manual. Safety features include over-heating protection and indicators.
This document provides specifications for an automatic water distillation unit. The unit distills water to generate reagent water type III with a maximum conductivity of 0.01 mS/cm at 25°C for use in washing and quantitative analysis. It has a stainless steel construction, 1.5 liter per hour capacity, operates on 220VAC power, and has automatic shutoff when water levels are low.
This document specifies a water geyser (heater) that has a 120 liter capacity, heats water to 90°C with thermostatic control, and requires a 220 VAC ±25%, 47 to 53 Hz power supply for operation in a laboratory setting.
A general purpose water bath has an inner size of approximately 0.4 x 0.3 x 0.1 meters, is made of stainless steel inside and stove enameled outside, and can maintain temperatures from ambient to 100°C with an accuracy of ±2°C. It has a stainless steel cover with 12 holes and features a drain cock or plug, double-walled insulation, and a pilot lamp to indicate thermostat operation.
This document describes the specifications of a water bath used for incubating culture tubes in coliform analysis. The water bath has an inner size of approximately 0.4 x 0.3 x 0.15 meters, is made of stainless steel inside and stove enameled outside, and can maintain temperatures from ambient to 50°C with an accuracy of ±0.1°C. It has a double wall for insulation, a dome lid to hold test tubes vertically, and displays the internal water temperature.
The document specifies the requirements for a wash bottle used to flush glassware. The wash bottle must be made of polythene, hold 500 ml of liquid, and have a bent nozzle and screw cap. It was last reviewed on October 23, 2007 and is used to wash away any sticking sediment from glassware.
The document specifies volumetric flasks that will be used for sediment analyses in a laboratory. The flasks must comply with IS 915-1975, be made of Corning glass or similar material, and come in sizes of 50, 100, 250, and 500 ml with B class accuracy.
The visual accumulation tube is used to assess particle sizes of sediments. It consists of a vertical transparent settling tube through which sediment samples are passed. Particles settle individually based on terminal velocity related to diameter, and accumulated sediment volume in the tube's narrow end relates to solid weight. It is best for uniform, spherical particles. The analysis involves introducing a small sample and recording accumulated sediment height over time.
This document provides specifications for a vacuum pump for general laboratory use. The pump is a single stage pump with a capacity of 50 l/min, capable of reaching a final vacuum of 0.05 mm Hg without ballast or 2 mm Hg with ballast. It has a 200W motor that operates on 220VAC at 47-53 Hz. Accessories include a filter, regulator, gauge, hose, and valve. The pump is designed for noise free operation.
This document provides specifications for a turbidity meter used to directly measure suspended matter in water samples. The turbidity meter has a range of 0 to 1000 NTU in at least 2 ranges, an accuracy of ±2% full scale deflection, and requires a power supply of 220 VAC ±25%, 47 to 53 Hz. Accessories for the turbidity meter include an ambient light shield, 6 spare tubes, a sensor stand, voltage stabilizer, instruction manual, and dust cover.
This tool kit contains basic tools for minor repairs of electrical laboratory equipment, including a set of screwdrivers, pliers, soldering iron, and multi-meter. The tools come in a lockable storage box for organization and security.
The document specifies the requirements for a microprocessor-controlled TOC analyzer. It must be able to directly measure total carbon, total organic carbon, and purgeable organic carbon in water samples. It uses high temperature catalytic combustion up to 900°C and non-dispersive infrared detection. It must have a measuring range of 1-500 mg/L carbon, precision within 3%, and detection limit of at least 500 ppb carbon. The analyzer and auto-sampler require 240V 50Hz power and it uses nitrogen or high purity air as carrier gases. It includes an auto-sampler, syringes, printer, manuals, spare parts, and application software in English.
This document provides specifications for a tissue grinder used to prepare tissue or sediment samples. The grinder must be able to macerate glass fibre filters and is a manual, porcelain device used to homogenize samples for further analysis such as chemical extraction.
This document specifies a set of thermometers for laboratory use, including mercury-filled glass thermometers with three temperature ranges (0-80oC, 10-150oC, 20-250oC) and an accuracy of ±0.5oC, along with a storage box for the thermometers.
This document specifies test tubes for laboratory use in sediment analysis. It outlines that the test tubes should be made of Corning glass or similar material, and lists two standard sizes - 15 x 125 mm and 25 x 200 mm in diameter and height. The test tubes are intended for general use in analyzing sediments in the laboratory.
The document specifies requirements for test sieves with a shaker. It requires stainless steel sieves that are 200mm in diameter, 50mm in height, and have nominal aperture sizes between 63-250 micrometers. It also requires a shaker capable of holding at least 5 sieves that runs on 220V power. Accessories include a sieve brush and wash bottle. Some sieves can be used manually without the shaker.
This document provides specifications for a magnetic stirrer with a hot plate. It can rotate between 0-1200 rpm and heat with a 300 Watt thermostatically controlled element. The stirrer has a stainless steel top and comes with PTFE coated magnets ranging from 10-50mm in 5mm increments, with two of each size included as accessories.
This document provides specifications for a sterilizer autoclave. The autoclave is 0.3 x 0.5 meters in size, made of stainless steel inside and lid with an enameled outside. It operates at a working pressure of 1 bar with a maximum pressure of 1.5 bars. Accessories include a pressure gauge, steam release cock, safety valve, perforated aluminum basket, water level indicator, and lifting arrangement. The autoclave is powered by 220 VAC at 47-53 Hz and uses approximately 2500 Watts.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Programming Foundation Models with DSPy - Meetup Slides
Final met handbook 180514
1. HP IIIndian Hydrology Project
Technical Assistance
(Implementation Support) and
Management Consultancy
Hydro-Meteorology Handbook:
Precipitation and Climate
May 2014
2. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Hydro-Meteorology Handbook: Precipitation and Climate
Issue and Revision Record
Revision Date Originator Checker Approver Description
0 21/05/14 Helen Houghton-Carr Version for approval
1
2
3
3. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page i
Contents
Contents i
Glossary iii
1. Introduction
1.1 HIS Manual
1.2 Other HPI documentation
1.3 Rainfall data in groundwater studies
1
2
3
4
2. The Data Management Lifecycle in HPII 5
2.1 Use of hydrological information in policy and decision-
making
2.2 Hydrological monitoring network design and development
2.3 Data sensing and recording
2.4 Data validation and archival storage
2.5 Data synthesis and analysis
2.6 Data dissemination and publication
2.7 Real-time data
5
6
6
6
7
8
8
3. Hydro-Meteorological Monitoring Stations and Data 10
3.1 Types of hydro-meteorological monitoring station
3.2 Hydro-meteorological monitoring networks
3.3 Site inspections, audits and maintenance
3.4 Data sensing and recording
3.5 Data processing
10
10
14
14
15
4. Rainfall Data Processing and Analysis 18
4.1 Data entry
4.2 Primary validation
4.3 Secondary validation
4.4 Correction and completion
4.5 Compilation
4.6 Analysis
18
21
24
29
33
37
5. Snow Data Processing and Analysis 40
5.1 Snow data in the Hydrology Project
5.2 Data entry
5.3 Primary validation
5.4 Secondary validation
5.5 Analysis
40
41
43
43
44
6. Climate Data Processing and Analysis 46
6.1 Data entry
6.2 Primary validation
6.3 Secondary validation
6.4 Correction and completion
6.5 Analysis
46
49
51
54
56
7. Data Dissemination and Publication 59
7.1 Hydro-meteorological products
7.2 Annual reports
7.3 Periodic reports
7.4 Special reports
59
59
62
63
4. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page ii
7.5 Dissemination to hydrological data users 63
References 64
Annex I States and agencies participating in the Hydrology Project 65
Annex II Summary of distribution of hard copy of HPI HIS Manual
Surface Water
66
Annex III Summary of distribution of hard copy of HPI HIS Manual
Groundwater
67
List of figures
1.1 Hydrometric information lifecycle 1
4.1 Definition of test and neighbouring stations 26
4.2 Definition sketch for double mass analysis 28
4.3 Example of basin area divided into Theissen polygons 35
4.4 Example of drawing isohyets using linear interpolation 35
List of tables
1.1 HPI hydro-meteorology training modules 4
2.1 Hydro-meteorological data processing timetable for data for
month n 8
3.1 Where to go in the HIS Manual SW for hydro-meteorological
data management guidance: rainfall and snow 11
3.1 cont/ Where to go in the HIS Manual SW for hydro-meteorological
data management guidance: climate and evaporation 12
4.1 Measurement errors for rainfall data 22
5.1 Measurement errors for snow data 44
6.1 Measurement errors for climate data 50
5. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page iii
Glossary
ADCP Acoustic Doppler Current Profiler
ARG Autographic Rain Gauge
AWS Automatic Weather Station
BBMB Bhakra-Beas Management Board
CGWB Central Ground Water Board
CPCB Central Pollution Control Board
CWC Central Water Commission
CWPRS Central Water and Power Research Station
Div Division
DPC Data Processing Centre
DSC Data Storage Centre
DWLR Digital Water Level Recorder
e-GEMS Web-based Groundwater Estimation and Management System
(HPII)
eHYMOS Web-based Hydrological Modelling System (HPII)
eSWDES Web-based Surface Water Date Entry System in e-SWIS (HPII)
e-SWIS Web-based Surface Water Information System (HPII)
FCS Full Climate Station
GEMS Groundwater Estimation and Management System (HPI)
GW Groundwater
GWDES Ground Water Data Entry System (HPI)
GWIS Groundwater Information System (GPI)
HDUG Hydrological Data User Group
HIS Hydrological Information System
HP Hydrology project (HPI Phase I, HPII Phase II)
HYMOS Hydrological Modelling System (HPI)
IMD India Meteorological Department
Lab Laboratory
MoWR Ministry of Water Resources
NIH National Institute of Hydrology
SRG Standard Rain Gauge
Stat Station
Sub-Div Sub-Division
SW Surface Water
SWDES Surface Water Data Entry System (HPI)
TBR Tipping Bucket Raingauge
ToR Terms of Reference
WISDOM Water Information System Data Online Management (HPI)
WQ Water Quality
7. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 1
1. Introduction
This Hydrology Project Phase II (HPII) Handbook provides guidance for the management of hydro-
meteorological data on rainfall, snow and other climate variables. The data are managed within a
Hydrological Information System (HIS) that provides information on the spatial and temporal
characteristics of the quantity and quality of surface water, including hydro-meteorology, and
groundwater. The information is tuned to the requirements of the policy makers, designers and
researchers to provide evidence to inform decisions on long-term planning, design and
management of water resources and water use systems, and for related research activities. The
Indian States and Central Agencies participating in the Hydrology Project are listed in Annex I.
However, this Handbook is also relevant to non-HP States.
It is important to recognise that there are two separate issues involved in managing hydro-
meteorological information. The first issue covers the general principles of understanding
monitoring networks, of collecting, validating and archiving data, and of analysing, disseminating
and publishing data. The second covers how to actually do these activities using the database
systems and software available. Whilst these two issues are undeniably linked, it is the first – the
general principles of data management - that is the primary concern. This is because improved
data management practices will serve to raise the profile of Central/State hydrometric agencies in
government and in the user community, highlight the importance of hydro-meteorological data for
the design of water-related schemes and for water resource planning and management, and
motivate staff, both those collecting the data and those in data centres.
This Handbook aims to help HIS users locate and understand documents relevant to hydro-
meteorology in the library available through the Manuals page on the Hydrology Project website.
The Handbook is a companion to the HIS manuals. The Handbook makes reference to the six
stages in the hydrometric information lifecycle (Figure 1.1), in which the different processes of data
sensing, manipulation and use are stages in the development and flow of information. The cycle
and associated HIS protocols are explored more fully in Section 2. Subsequent sections cover
different stages of the cycle for different hydro-meteorological variables.
Figure 1.1 Hydrometric information lifecycle (after: Marsh, 2002)
8. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 2
1.1 HIS Manual
The primary reference sources are the HIS Manual Surface Water (SW) and HIS Manual
Groundwater (GW), two of many hundreds of documents generated during Hydrology Project
Phase I (HPI) to assist staff working in observation networks, laboratories, data processing centres
and data communication systems to collect, store, process and disseminate hydrometric data and
related information. During HPI, special attention was paid to the standardisation of procedures for
the observation of variables and the validation of information, so that it was of acceptable quality
and compatible between different agencies and States, and to facilities for the proper storage,
archival and dissemination of data for the system, so that it was sustainable in the long-term.
Therefore, the majority of the documents produced under HPI, particularly those relating to
fundamental principles, remain valid through and beyond HPII. Some parts of the guides, manuals
and training material relating to HPI software systems (SWDES, HYMOS, WISDOM, GWDES,
GEMS, GWIS) have been partially or wholly superseded as replacement Phase II systems (e-
GEMS, e-SWIS) become active.
The HIS Manual SW and HIS Manual GW describe the procedures to be used to arrive at a sound
operation of the HIS in regard to rainfall, snow and climate data. The HIS Manual SW and HIS
Manual GW each consist of 10 volumes. Each volume contains one or more of the following
manuals, depending on the topic:
• Design Manual (DM) - procedures for the design activities to be carried out for the
implementation and further development of the HIS.
• Field Manual (FM) or Operation Manual (OM) – detailed instructions describing the activities to
be carried out in the field (station operation, maintenance and calibration), at the laboratory
(analysis), and at the Data Processing Centres (data entry, validation, processing,
dissemination, etc). Each Field/Operation Manual is divided into a number of parts, where
each part describes a distinct activity at a particular field station, laboratory or data processing
centre.
• Reference Manual (RM) - additional or background information on topics dealt with or
deliberately omitted in the Design, Field and Operation Manuals.
Those HIS Manual SW/GW volumes relevant to rainfall and climate are:
SW/GW Volume 1: Hydrological Information System: a general introduction to the HIS, its
structure, HIS job descriptions, Hydrological Data User Group (HDUG) organisation and user data
needs assessment. The content of the SW and GW volumes is identical.
• Design Manual
• Field Manual
Part II: Terms of Reference for HDUG
Part III: Data needs assessment
SW/GW Volume 2: Sampling Principles: units, principles of sampling in time and space and
sampling error theory. The content of the SW and GW volumes is identical.
• Design Manual
SW/GW Volume 3: Hydro-meteorology: network design, implementation, operation and
maintenance. The content of the SW and GW volumes is identical.
• Design Manual
• Field Manual
Part I: Network design and site selection
Part II: Standard raingauge station (SRG) operation and maintenance
9. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 3
Part III: Autographic raingauge station (ARG or TBR (and SRG)) operation and
maintenance
Part IV: Full climate station (FCS) operation and maintenance
Part V: Field inspections, audits maintenance and calibration
• Reference Manual
SW Volume 8: Data processing and analysis: specification of procedures for Data Processing
Centres (DPCs).
• Operation Manual
Part I: Data entry and primary validation
Part II: Secondary validation
Part III: Final processing and analysis
Part IV: Data management
GW Volume 8: Data processing and analysis
• Operation Manual
Part V: Groundwater Year Book
SW Volume 10: Surface Water protocols: outline of protocols for data collection, entry, validation
and processing, communication, inter-agency validation, data storage and dissemination, HIS
training and management.
• Operation Manual
Data entry forms
In this Handbook, individual parts of the HIS Manual SW/GW are referred to according to the
nomenclature “SW/GWvolume-manual(part)” e.g. SW Volume 3: “Hydro-meteorology” Field
Manual Part II: “Standard raingauge station (SRG) operation and maintenance” is referred to as
SW3-FM(II), and GW Volume 8: “Data processing and analysis” Operation Manual Part V:
“Groundwater Year Book” is referred to as GW8-OM(V).
A hard copy of the relevant manuals should be available for the locations listed in Annex II. For
example, a hard copy of SW3-FM(II) should be available at all meteorological stations where
rainfall measurement with an SRG takes place. Similarly, SW8-OM(I) should be available at all
Data Processing Centres where data entry and primary validation take place.
As noted, there is some inevitable overlap and repetition between the HIS Manual SW and the HIS
Manual GW (e.g. Volume 3). In the following sections of this Handbook, reference is generally
made only to the HIS Manual SW, as the majority of hydro-meteorological reference material is
incorporated in here, unless there is important additional information in the HIS Manual GW.
1.2 Other HPI documentation
Other HPI documents of relevance to hydro-meteorology include:
• The e-SWIS software manual, and the SWDES and HYMOS software manuals - although
SWDES and HYMOS are being superseded by e-SWIS in HPII, to promote continuity, e-SWIS
contains eSWDES and eHYMOS modules.
• “Illustrations: hydrological observations” – an illustrative booklet demonstrating how to make
measurements of rainfall, water level and flow at stations, and also how to carry out an
inspection at those stations.
10. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 4
Table 1.1 HPI hydro-meteorology training modules
Topic Module Title
Meteorology 07 How to make data entry for rainfall data
08 How to carry out primary validation for rainfall data
09 How to carry out secondary validation of rainfall
10 How to correct and complete rainfall data
11 How to compile rainfall data
12 How to analyse rainfall data
13 How to report on rainfall data
14 MISSING – How to process evaporation data
15 How to make data entry for climatic data
16 How to carry out primary validation for climatic data
17 How to carry out secondary validation of climatic data
18 MISSING – How to correct and complete climatic data
19 How to analyse climatic data
20 How to report on climatic data
Hydrometry 43* Statistical Analysis with Reference to Rainfall & Discharge Data
44* How to carry out correlation and spectral analysis
45* How to review Monitoring Networks
* Hydrometry modules also relevant to Hydro-Meteorology. 43 and 44 present statistical analysis techniques
as applied to, say, analyse rainfall data.
• “Surface Water O&M norms” – a maintenance guide for hydro-meteorology, stage-discharge
and water quality instrumentation and equipment.
• “Surface Water Yearbook” – a template for a Surface Water Yearbook published at State level.
• Hydro-meteorology training modules – these relate to the entry, primary and secondary
validation, processing, analysis and reporting of rainfall and climate data using SWDES and
HYMOS (see Table 1.1). Their contents have been largely incorporated into this Handbook as
the underlying principles for data validation and analysis remain valid.
1.3 Rainfall data in groundwater studies
Access to rainfall data is important in interpretation of groundwater level data, and for balancing
recharge, discharge and storage of groundwater systems. GW3-DM, FM and RM describe the
design, implementation, operation and maintenance of hydrometeorological networks, and rainfall
data may be stored in the groundwater data processing and analysis software e-GEMS. However,
subsequent data processing and analysis of rainfall data are covered only in SW8-OM and the
surface water software e-SWIS has a wider range of validation and manipulation tools for rainfall
data than e-GEMS. Therefore, it is recommended to carry out the majority of rainfall data
processing and analysis in e-SWIS, and then export final datasets from e-SWIS, for import to e-
GEMS.
11. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 5
2. The Data Management Lifecycle in HPII
Agencies and staff with responsibilities for hydrometric data have a pivotal role in the development
of hydro-meteorological information, through interacting with data providers, analysts and policy
makers, both to maximise the utility of the datasets and to act as key feedback loops between data
users and those responsible for data collection. It is important that these agencies and staff
understand the key stages in the hydrometric information lifecycle (Figure 1.1), from monitoring
network design and data measurement, to information dissemination and reporting. These later
stages of information use also provide continuous feedback influencing the overall design and
structure of the hydrometric system. While hydrometric systems may vary from country to country
with respect to organisation set-ups, observation methods, data management and data
dissemination policies, there are also many parallels in all stages of the cycle.
2.1 Use of hydro-meteorological information in policy and decision-making
The objectives of water resource development and management in India, based on the National
Water Policy and Central/State strategic plans, are: to protect human life and economic functions
against flooding; to maintain ecologically-sound water systems; and to support water use functions
(e.g. drinking water supply, energy production, fisheries, industrial water supply, irrigation,
navigation, recreation, etc). These objectives are linked to the types of data that are needed from
the HIS. SW1-DM Chapter 3.3 presents a table showing HIS data requirements for different use
functions on page 19. In turn, these use functions lead to policy and decision-making uses of HIS
data, such as: water policy, river basin planning, water allocation, conservation, demand
management, water pricing, legislation and enforcement.
Hence, freshwater management and policy decisions across almost every sector of social,
economic and environmental development are driven by the analysis of hydrometric information.
Its wide-ranging utility, coupled with escalating analytical capabilities and information dissemination
methods, have seen a rapid growth in the demand for hydrometric data and information over the
first decades of the 21st century. Central/State hydrometric agencies and international data
sharing initiatives are central to providing access to coherent, high quality hydrometric information
to a wide and growing community of data users. Hydrological data users may include water
managers or policymakers in Central/State government offices and departments, staff and
students in academic and research institutes, NGOs and private sector organisations, and
hydrology professionals. An essential feature of the HIS is that its output is demand-driven, that is,
its output responds to the hydrological data needs of users.
SW1-FM(III) presents a questionnaire for use when carrying out a data needs assessment to
gather information on the profile of data users, their current and proposed use of surface water,
groundwater, hydro-meteorology and water quality data, their current data availability and
requirements, and their future data requirements. Data users can, through Central/State
hydrometric agencies, play a key role in improving hydrometric data, providing feedback
highlighting important issues in relation to records, helping establish network requirements and
adding to a centralised knowledge base regarding national data. By embracing this feedback from
the end-user community, the overall information delivery of a system can be improved.
A key activity within HPII was a move towards greater use of the HIS data assembled under HPI.
Two examples of the use of HIS data include the Purpose-Driven Studies (PDS) and the Decision
Support Systems (DSS) components of HPII. See the Hydrology Project website for more
information about DSS and PDS, and access to PDS reports.
The 38 PDS, which were designed, prepared and implemented by each of the Central/State
12. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 6
hydrometric agencies, are small applied research projects to investigate and address a wide range
of real-world problems and cover surface water, groundwater, hydro-meteorology and water quality
topics. Some examples of projects include optimisation of the river gauging station and raingauge
networks in Maharashtra (PDS number SW-MH-1), and a snowmelt runoff study in the Beas basin
(PDS number SW-NIH-1). The PDS utilise hydrometric data and products developed under HPI,
supplemented with new data collected during HPII.
Two separate DSS programmes were set up under HPII. One, for all participating implementing
agencies, called DSS Planning (DSS-P), has established water resource allocation models for
each State to assist them to manage their surface and groundwater resources more effectively.
The other, called DSS Real-Time (DSS-RT) was specifically for the Bhakra-Beas Management
Board (BBMB), although a similar DSS-RT study has also now been initiated on the Bhima River in
Maharashtra. The DSS programmes have been able to utilise hydrological data assembled under
the Hydrology Project to guide operational decisions for water resource management.
2.2 Hydro-meteorological monitoring network design and development
Section 3.2 of this Handbook outlines the design and development of hydro-meteorological
monitoring networks. Networks are planned, established, upgraded and evolved to meet a range
of needs of data users and objectives, most commonly water resources assessment and
hydrological hazard mitigation (e.g. flood forecasting). It is important to ensure that the hydro-
meteorological, surface water, groundwater and water quality monitoring networks of different
agencies are integrated as far as possible to avoid unnecessary duplication. In particular, a
raingauge network should have sufficient spatial coverage that all flow monitoring stations are
adequately covered. Integration of networks implies that networks are complimentary and that
regular exchange of data takes place to produce high quality validated datasets. Responsibility for
maintenance of Central/State hydrometric networks is frequently devolved to a regional (Divisional)
or sub-regional (Sub-Divisional) level.
2.3 Data sensing and recording
Sections 3.1 to 3.4 of this Handbook review hydro-meteorological monitoring networks and
stations, maintenance requirements and measurement techniques. Responsibility for operation of
Central/State hydro-meteorological monitoring stations is frequently devolved to a regional
(Divisional) or sub-regional (Sub-Divisional) level. However, it is important that regular liaison is
maintained between sub-regions and the Central/State agencies through a combination of field site
visits, written guidance, collaborative projects and reporting, in order to ensure consistency in data
collection and initial data processing methods across different sub-regions, maintain strong
working relationships, provide feedback and influence day-to-day working practice. Hence, the
Central/State agencies are constantly required to maintain a balance of knowledge between a
broad-scale overview and regional/sub-region hydro-meteorological awareness. Operational
procedures should be developed in line with appropriate national and international (e.g. Indian,
ISO, WMO) standards (e.g. WMO Report 168 “Guide to Hydrological Practices”).
For the Hydrology Project, field data from observational stations are required to be received at
Sub-Divisional office level by the 5th
working day of the following month (SW10-OM Protocols and
Procedures).
2.4 Data validation and archival storage
The quality control and long-term archiving of hydro-meteorological data represent a central
function of Central/State hydrometric agencies. This should take a user-focused approach to
13. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 7
improving the information content of datasets, placing strong emphasis on maximising the final
utility of data e.g. through efforts to improve completeness and fitness-for-purpose of
Centrally/State archived data. Section 3.5 of this Handbook summarises the stages in the
processing of hydro-meteorological data. Sections 4 to 6 of this Handbook cover the process from
data entry through primary and secondary validation to correction and completion of data, and also
compilation and analysis of data (Section 2.5), for rainfall, snow and climate data, respectively.
During all levels of validation, staff should be able to consult station metadata records detailing the
history of the site and its hydro-meteorological performance, along with topographical and isohyetal
maps and previous quality control logs. Numerical and visual tools available at different phases of
the data validation process, such as versatile hyetograph plotting and manipulation software to
enable comparisons between different near-neighbour rainfall measurement sites, assessment of
basin rainfall input hyetographs and assessment of time series statistics greatly facilitate validation.
High-level appraisal by Central/State staff, examining the data in a broader spatial context, can
provide significant benefits to final information products. It also enables evaluation of the
performance of sub-regional data providers, individual stations or groups of stations, which can
focus attention on underperforming sub-regions and encourage improvements in data quality.
A standardised data assessment and improvement procedure safeguards against reduced quality,
unvalidated and/or unapproved data reaching the final data archive from where they can be
disseminated. However, Marsh (2002) warns of the danger of data quality appraisal systems that
operate too mechanistically, concentrating on the separate indices of data quality rather than the
overall information delivery function.
For the Hydrology Project, the timetable for data processing is set out in SW10-OM Protocols and
Procedures, and summarised in Table 2.1 of this Handbook. Data entry and primary validation of
field data from observational stations is required to be completed at Sub-Divisional office level by
the 10th
working day of the following month (e.g. for June data by 10th
working day in July), ready
for secondary validation by State offices. Initial secondary validation, in State DPCs for State data,
and IMD local offices for IMD data, should be completed by the end of that month (e.g. for June
data by 31st
July). Some secondary validation will not be possible until the end of the hydrological
year when the entire year’s data can be reviewed in a long-term context, and compared with IMD
data, so data should be regarded as provisional approved data until then (e.g. for June data by the
end of the hydrological year plus 3 months), after which data should be formally approved and
made available for dissemination to external users. At certain times of year (e.g. during the
monsoon season), this data processing plan may need to be compressed, so that validated hydro-
meteorological data are available sooner.
2.5 Data synthesis and analysis
Central/State hydrometric agencies play a key role in the delivery of large-scale assessments of
rainfall data and other climate data. Through their long-term situation monitoring, they are often
well placed to conduct or inform scientific analysis at a State, National or International level, and
act as a source of advice on data use and guidance on interpretation of precipitation patterns. This
is especially true in the active monitoring of the State or National situation or the assessment of
conditions at times of extreme events (e.g. monsoonal rains, droughts) where agencies may be
asked to provide input to scientific reports and research projects, as well as informing policy
decisions, media briefings, and increasing public understanding of the state of the water
environment. Sections 4 to 6 of this Handbook cover compilation and analysis of data, as well as
the process from data entry through primary and secondary validation to correction and completion
of data (Section 2.4), for rainfall, snow and climate data, respectively.
14. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 8
Table 2.1 Hydro-meteorological data processing timetable for data for month n
Activity Responsibility Deadline
Rainfall, snow and climate data
Data receipt Sub-Divisional office 5
th
working day of month n+1
Data entry Sub-Divisional/Divisional office 10
th
working day of month n+1
Primary validation Sub-Divisional/Divisional office 10
th
working day of month n+1
Secondary validation State DPC
State DPC
Initial - end of month n+1
Final – end of hydrological year +
3 months
Correction and completion State DPC
State DPC
Initial - end of month n+1
Final – end of hydrological year +
3 months
Compilation State DPC As required
Analysis State DPC As required
Reporting State DPC At least annually
Data requests State DPC 95% - within 5 working days
5% - within 20 working days
Interagency validation IMD At least 20% of State stations, on
rolling programme, by end of
hydrological year + 6 months
2.6 Data dissemination and publication
One of the primary functions of Central/State hydrometric agencies is to provide comprehensive
access to information at a scale and resolution appropriate for a wide range of end-users.
However, improved access to data should be balanced with a promotion of responsible data use
by also maintaining end-user access to important contextual information. Thus, the dissemination
of user guidance information, such as composite summaries that draw users’ attention to key
information and record caveats (e.g. monitoring limitations, high levels of uncertainty regarding
specific rainfall event accuracy, major changes in hydro-meteorological setup), is a key
stewardship role for Central/State hydrometric agencies, as described in Section 7 of this
Handbook.
For large parts of the 20th century the primary data dissemination route for hydrometric data was
via annual hardcopy publications of data tables i.e. yearbooks. However, the last decade or so has
seen a shift towards more dynamic web-based data dissemination to meet the requirement for
shorter lag-time between observation and data publication and ease of data re-use. Like many
countries, India now uses an online web-portal as a key dissemination route for hydrometric data
and associated metadata which provides users with dynamic access to a wide range of information
to allow selection of stations. At least 95% of data requests from users should be processed within
5 working days. More complex data requests should be processed within 20 working days.
2.7 Real-time data
During HPII many implementing agencies developed low cost real-time data acquisition systems,
feeding into bespoke databases and available on agency websites. Such systems often utilise
short time interval recording of data e.g. 5 minutes, 15 minutes, etc. In some instances, agencies
are taking advantage of the telemetry aspect of real-time systems as a cost-effective way of
acquiring data from remote locations. However, for some operational purposes (e.g. real-time
flood forecasting, reservoir operation, etc), real-time data may need to be used immediately.
15. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 9
Real-time data should go through some automated, relatively simple data validation process before
being input to real-time models e.g. checking that each incoming data value is within pre-set limits
for the station, and that the change from preceding values is not too large. Where data fall outside
of these limits, they should generally still be stored, but flagged as suspect, and a warning
message displayed to the model operators. Where suspect data have been identified, a number of
options are available to any real-time forecasting or decision support model being run, and the
choice will depend upon the modelling requirements. Whilst suspect data could be accepted and
the model run as normal, it is more common to treat suspect data as missing or to substitute them
with some form of back-up, interpolated or extrapolated data. This is necessary for hydrometric
agencies to undertake some of their day-to-day functions and, in such circumstances, all the data
should be thoroughly validated as soon as possible, according to the same processing timetable
and protocols as other climate data.
Real-time data should also be regularly transferred to the e-SWIS database system, through
appropriate interfaces, in order to ensure that all hydro-meteorological data are stored in a single
location and provide additional back-up for the real-time data, but also to provide access to the
data validation tools available through the eSWDES and eHYMOS modules of e-SWIS.
16. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 10
3. Hydro-Meteorological Monitoring Stations and Data
3.1 Types of hydro-meteorological monitoring station
SW3-FM(I) Chapter 2.1 lists different types of hydro-meteorological stations and instruments which
measure various rainfall and climate variables. For each of these, Table 3.1 (two parts) lists the
relevant section in the HIS Manual SW for detailed information on design and installation,
maintenance, measurement, data entry, primary and secondary validation, correction and
completion of data, compilation and analysis of data, and reporting. Stations include:
• SRG – a rainfall station equipped with a standard or non-recording raingauge. Additional
information on design and installation (Table 3.1) includes the appropriate capacity of
raingauge containers for different Indian States.
• ARG – a rainfall station with an autographic or recording raingauge, which will also have an
SRG for check purposes.
• TBR – a tipping bucket raingauge is a type of ARG, often connected to a data logger, which will
also have an SRG for check purposes. Additional information on design and installation (Table
3.1) includes the relative advantages of ARGs and TBRs, compared to each other and to
SRGs.
• Snow stations – a type of station not included in HPI, where observation are made of:
Snowfall since the last observation;
Total depth of snow on the ground (i.e. the depth of the snowpack);
Snow-water equivalent (SWE i.e. the depth of liquid precipitation contained in that snowfall
and/or the snowpack).
• FCS – a full climate station where, in addition to rainfall, a comprehensive range of other
climate variables are observed for direct measurement of evaporation and/or for indirect
estimation of evaporation:
Pan-evaporation (direct measurement) using a pan-evaporimeter;
Temperature of pan water;
Sunshine duration using a sunshine recorder;
Air temperature using thermometers and optional thermograph;
Humidity using thermometers and optional hygrograph;
Wind speed and direction using an anemometer and wind vane;
Atmospheric pressure using a barometer and/or barograph.
• AWS – an automatic weather station is an FCS where the climate variables are observed by
automatic/recording means. These were not included in HPI documentation. AWS were not
included in HPI.
A set of specifications for hydrometric equipment was compiled under HPI and updated under
HPII. The specifications, which are downloadable from the Hydrology Project website, provide a
guideline for procurement (with examples of some procurement templates and documents also on
the Hydrology Project website).
3.2 Hydro-meteorological monitoring networks
Monitoring networks should be considered to be dynamic entities. It is important that the current
utility of well-established monitoring networks is periodically assessed to ensure that they continue
17. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 11
Table 3.1 Where to go in the HIS Manual SW for hydro-meteorological data management guidance: rainfall and snow
Instrument
/ Variable
Design &
Installation
Maintenance Measurement Data entry Primary
Validation
Secondary
Validation
Correction &
Completion
Compilation Analysis Reporting
SRG SW3-DM
6.2.1, 8.2.2
SW3-FM(II)
1.3
SW3-FM(V)
2.2, 3.2
SW3-FM(II)
1.2
SW8-OM(I)
4.4, 4.5
SW8-OM(I)
5
SW8-OM(II)
2
SW8-OM(II)
3
SW8-OM(II)
4
SW8-OM(III)
4
SW8-OM(III)
9
ARG SW3-DM
6.2.2, 8.2.3
SW3-FM(III)
2.3
SW3-FM(V)
2.2, 3.3
SW3-FM(III)
2.2.2
SW8-OM(I)
4.6
SW8-OM(I)
5
SW8-OM(II)
2
SW8-OM(II)
3
SW8-OM(II)
4
SW8-OM(III)
4
SW8-OM(III)
9
TBR SW3-DM
6.2.3, 8.2.4
SW3-FM(III)
3.3
SW3-FM(V)
2.2, 3.4
SW3-FM(III)
2.3.2
SW8-OM(I)
4.7
SW8-OM(I)
5
SW8-OM(II)
2
SW8-OM(II)
3
SW8-OM(II)
4
SW8-OM(III)
4
SW8-OM(III)
9
Snow Not included in HPI
19. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 13
to meet changing requirements and to optimise the information they deliver. Network reviews
should be done in collaboration with other agencies. SW3-FM(I) Chapter 1 and SW3-DM Chapter
3 describe network design and optimisation for monitoring rainfall and other climate variables. This
is a multi-step process comprising:
1. Identification of hydrological data users and their data needs to understand what data are
required and at what frequency.
2. Definition of the purposes and objectives of the network in order to fulfill the hydrological data
need, and evaluation of the consequences of not meeting those targets, to inform a
prioritisation of objectives in case of budget constraints.
3. Evaluation of the existing and required network densities, using an effectiveness measure
which takes into account the spatial and temporal correlation of the variables. This step and
steps 4 and 5 may involve the development of regionalisation and network optimisation
techniques (e.g. Institute of Hydrology, 1999; Hannaford et al., 2013).
4. Evaluation of the existing network versus the required one in relation to network density,
purposes and objectives, distribution with respect to surface water and groundwater monitoring
networks, adequacy of existing equipment and operational procedures, and possible
improvements to existing network.
5. Site and equipment selection i.e. the identification of gaps in the existing network if it is
inadequate to meet the purposes and objectives. This may require the collection of maps and
background information to inform the revised network design.
6. Estimation of overall costs of installing, operating and maintaining the existing and new sites.
Achieving an optimum network design may involve an iterative process, repeating some or all
of steps 3 to 6, until a satisfactory outcome is reached.
7. Preparation of phased implementation plan for optimum network that is prioritised, realistic and
achievable in the time scales allowed.
8. Selection of sites. SW3-FM(I) devotes Chapter 2 to this topic, identifying the factors that
should be taken into consideration to ensure long-term reliable data. These include: technical
(positioning to minimise estimation errors and optimise integration with surface water and
groundwater networks); environmental (topography around site, exposure conditions at site,
future development near site, vegetation at and near site, proximity of water bodies, no water-
logging at site); logistical (accessibility, communication, staffing); security (location of site,
design of site, staffing); legal (land acquisition, rights of passage); and financial (cost of land,
cost of civil works, equipment procurement, calibration and maintenance, operating costs,
staffing) aspects. Site selection should be carried out in collaboration with IMD and should
involve a site visit, which may reveal that the desired location is unsuitable, and an alternative
site may need to be considered.
The selection of appropriate locations for snow stations (also known as snow courses) may be
challenging because of terrain and wind effects. In flat, open areas, it is desirable to have
snow stations in typical landscapes, such as in open fields and forests, with different snow
accumulation conditions. In mountainous areas, additional criteria may apply:
• At sites sufficiently accessible to ensure continuity of surveys
• At elevations and exposures where there is little or no melting prior to the peak
accumulation
• At a site having protection from strong wind movement
20. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 14
• In forested areas where the sites can be located in open spaces sufficiently large so that
snow can fall to the ground without being intercepted by the trees
9. Establishment of a framework for periodic network reviews (e.g. after 3 years or sooner if new
data needs develop) i.e. starting this process again from step 1.
As an example of the theory and practical aspects of network design, SW3-RM presents a pilot
study for designing a raingauge network for two sub-basins of the Mahanadi river basin in Orissa.
A good example of a monitoring network review under HPII is the Purpose Driven Study (PDS) on
optimisation of the river gauging station and raingauge networks in Maharashtra (PDS number
SW-MH-1).
For more detailed information see: SW2-DM Chapter 7 which provides some generic guidance on
types of network and the steps in network design; SW2-DM Chapters 3.2.1 to 3.2.6 which describe
classification of stations and offer some examples of types of network; and Hydro-Meteorology
Training Module 45 “How to review monitoring networks”.
3.3 Site inspections, audits and maintenance
Regular maintenance of equipment, together with periodic inspections and audits, ensures
collection of good quality data and provides information that may assist in future data validation
queries. Table 3.1 lists the relevant section in the HIS Manual SW for maintenance of the different
types of hydro-meteorological stations and instruments. Whilst this topic is largely covered in
different parts of SW3-FM(II)-(V), information is collated together in the document “Surface Water
O&M norms” which is a maintenance guide for hydro-meteorology, stage-discharge and water
quality instrumentation and equipment.
Maintenance and calibration requirements depend to a large extent of the type of station,
instruments and equipment so are often site-specific. A supply of appropriate spare parts should
be kept on site and/or taken on station visits in case they are needed. SW3-FM(V) Annex II lists
maintenance norms for hydro-meteorological stations, including maintenance of civil works,
maintenance of equipment, costs of consumable items and payments to staff (where the costs
should be regarded as out of date).
Formal inspections cum audits are carried out, and station log sheets completed, at a frequency
dependent on the importance of the station, the type of station and the time of year and will
typically vary between monthly and annually as set out in SW3-FM(V) Chapter 1, with station log
sheets for inspections of rainfall and climate stations in FM(V) Annex I. Activities may include:
checking the performance of and motivating the field staff; and identifying existing or potential
problems with the site, instruments, equipment and observation procedures at an early stage so
they can be rectified. However, a brief inspection of the site and instrumentation should be made
every day that somebody is on site.
3.4 Data sensing and recording
Table 3.1 lists the relevant section in the HIS Manual SW for operational instructions on the
measurement of rainfall and other climate variables at hydro-meteorological stations. Note that
there is some overlap between SW3-FM and SW3-DM, and between the network design and site
selection topic (covered in Section 3.2 of this Handbook) and data measurement. See also the
document “Illustrations: hydrological observations” which demonstrates how to make
measurements of rainfall, water level and flow at stations, and also how to carry out an inspection
at those stations.
21. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 15
At an FCS, the observations of the various climate variables are made once or twice a day in a
prescribed sequence commencing from 10 minutes preceding the scheduled hours i.e. 08:20 for
08:30 readings, and 17:20 for 17:30 readings. The sequence is: wind instruments, raingauges,
thermometers, evaporation, radiation, and culminating with atmospheric pressure at exactly 08:30
and 17:30. Charts on autographic instruments are changed daily during the 08:30 reading, except
the sunshine recorder card which is changed during the 17:30 reading or after sunset, whichever is
later. Hourly values are abstracted from autographic charts and tabulated daily. Observers at an
FCS should also make observations of the depth of snow on the ground, should it occur.
For stations with only an SRG, the measurement is made at 08:30 IST only, though more frequent
observations are required during heavy rain to avoid overflow due to the limited capacity of the
raingauge container. SRGs at FCSs are read twice daily along with the other climate variables.
ARG chart recorders are digitised at 1-hour or 15-minute time intervals depending on what is most
appropriate for the location and the intensity of the rainfall (SW3-DM Chapter 5.2). TBRs with data
loggers can operate in time mode where the number of tips in a pre-set time interval (e.g. 1 hour,
15 minutes, etc) are recorded, or in event mode where the times of every tip are recorded, thereby
producing a more flexible record for subsequent analysis. However, note that event mode data
cannot currently be stored in e-SWIS.
At snow stations, observations are made of the snowfall since the last observation, the total depth
of snow on the ground (i.e. the depth of the snowpack) and the snow-water equivalent (SWE i.e.
the depth of liquid precipitation contained in that snowfall and/or the snowpack). Measurements
may be made daily or sub-daily. The accuracy of measurements of snowfall, snow depth and
SWE depends on the graduations of the scales being used, and on instrumental and subjective
errors. At some snow stations, data are augmented by regular measurements of sunshine,
temperature, humidity, wind speed and direction, and atmospheric pressure. The extent of snow
cover is usually made from one or a combination of field observations, aerial survey data and
satellite imagery.
The observer should always note any occurrences which may influence the climate variables as
observed by the instruments. These may include: damage to the equipment for a specified reason.
The observer should also note any maintenance activities carried out at the monitoring site (e.g.
change batteries, clean sensor, etc).
The observer should double-check that that any manual reading is taken correctly, and transcribed
correctly (e.g. decimal point in right place). If the reading is later transferred to another document
(e.g. hand copied or typed in, or abstracted from a chart), the observer should always check that
this has been done correctly. An experienced and suitably qualified observer should compare
measurements with equivalent ones from earlier that day or from the day before, if available, as an
additional form of checking. However, the observer should not, under any circumstances,
retrospectively alter earlier readings or adjust current readings, but should simply add an
appropriate comment.
Data collected in the field are delivered to a Data Processing Centre (DPC) on a variety of media,
including handwritten forms and notebooks, charts and digital data.
3.5 Data processing
SW8-OM(IV) Chapter 2 sets out the steps in processing of hydro-meteorological data, which starts
with preliminary checking in the field, as described in Section 3.4 of this Handbook, through receipt
of raw field data at a DPC, through successively higher levels of validation in State and Central
DPCs, before data are fully validated and approved in the National database. Validation ensures
that the data stored are as complete and of the highest quality as possible by: identifying errors
and sources of errors to mitigate them occurring again, correcting errors where possible, and
22. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 16
assessing the reliability of data. It is important for staff to be aware of the different errors that may
occur as described in SW8-OM(IV) Chapter 2.5.1.
Hydro-meteorological data validation is split into two stages: primary and secondary. Validation is
very much a two-way process, where each step feeds back to the previous step any comments or
queries relating to the data provided. The diverse hydrological environments found in India mean
that staff conducting data validation should be familiar with the expected climate in order to identify
potentially anomalous behaviour. SW3-RM presents some tabular and graphical summary data for
selected coastal and hill FCSs in the HPI Indian States for the period 1931-60, to give an insight
into typical annual variation of the climate variables. The data processing steps comprise:
1. Receipt of data according to prescribed target dates. Rapid and reliable transfer of data is
essential, using the optimal method based on factors such as volume, frequency, speed of
transfer/transmission and cost. Maintenance of a strict time schedule is important because it
gives timely feedback to monitoring sites, it encourages regular exchanges between field staff,
Sub-Divisional offices, State and Central agencies, it creates continuity of processing activities
at different offices, and it ensures timely availability of final (approved) data for use in policy
and decision-making.
2. Entry of data to computer, using the eSWDES module of e-SWIS, is primarily done at a Sub-
Divisional office level where staff are in close contact to field staff who have made the
observations and/or collected the chart or digital data. Historical data, previously only available
in hardcopy form, may also be entered this way. Each Central/State agency should have a
programme of historical data entry.
3. Primary data validation which should be carried out in State DPCs for State data and IMD local
offices for IMD data, as soon as possible after the observations are made, data extracted from
charts, or data downloaded from loggers, using the eSWDES module of e-SWIS. This ensures
that any obvious problems (e.g. indicating an instrument malfunction, observer error, etc) are
spotted at the earliest opportunity and resolved. Other problems may not become apparent
until more data have been collected, and data can be viewed in a longer temporal context
during secondary validation.
4. Secondary data validation which should be carried out in State DPCs for State data and IMD
local offices for IMD data, to take advantage of the information available from a large area by
focusing on comparisons with the same variable at other good quality, nearby monitoring sites
(analogue stations) which are expected to exhibit similar hydrological behaviours (e.g.
comparison of cumulative rainfall from two raingauges), uses the eHYMOS module of e-SWIS.
States should have access to IMD data during secondary validation, and may receive support
from IMD in this activity.
5. Data correction and completion are elements of data validation which are used to infill missing
value, sequences of missing values or correct clearly erroneous values, in order to make the
time series as complete as possible. Some suspect (doubtful) data values may still justifiably
remain after this stage if correction is not possible so that the original data value remains the
best estimate of the true value for that day and time.
6. Data storage. The e-SWIS HIS database, of both approved data and unapproved data
undergoing primary and secondary validation, is backed up automatically. Therefore, there is
no need to make regular back-ups, unless any data are stored outside the HIS database, for
instance in Excel files or other formats awaiting data entry, or in stand-alone real-time
databases – such files should be securely backed up, ideally onto an external back-up device
and/or backed up network server, so that there is no risk of data loss. All PCs should have up-
to-date anti-virus software.
23. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 17
Raw field data, in the form of handwritten forms and notebooks, and charts should also be
stored in a secure manner after database entry to ensure that original field data remain
available should any problems be identified during validation and analysis. Such hardcopy
data should ultimately be securely archived, in the State DPC for State data or IMD local office
for IMD data, possibly by scanning documents and storing them digitally.
7. Interagency data validation by IMD – IMD should aim to validate at least 20% of current and
historic data from State hydro-meteorological monitoring stations every year, on a rolling
programme, so that IMD has independently validated the data from every State gauge at least
once every 5 years. Interagency validation is a 2-way process and IMD should discuss any
identified issues and agree final datasets with State DPCs through a 2-way consultative
process, to build capacity for data validation within the States.
For rainfall, snow and climate data, Sections 4 to 6 of this Handbook, respectively, cover the
process from data entry through primary and secondary validation to correction and completion of
data, and also compilation (i.e. the transformation of data observed at one time interval to another
time interval e.g. aggregation from daily rainfall to monthly rainfall) and analysis of data.
24. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 18
4. Rainfall Data Processing and Analysis
4.1 Data entry
4.1.1 Overview
Entry of data to computer is primarily done at a Sub-Divisional office level where staff are in close
contact to field staff who have made the observations and/or collected the chart or digital data.
Data entry is carried out using e-SWIS, the data entry module of which replicates the SWDES
software from HPI, and is referred to as eSWDES. Prior to entry to computer, two manual
activities are essential: registration of receipt of the data, and manual inspection of the rainfall
charts, forms and notebooks from the field, for complete information and obvious errors. Data
entry (see Table 3.1) and primary validation of field data from observational stations is required to
be completed at Sub-Divisional office level by the 10th
working day of the following month (e.g. for
June data by 10th
working day in July), ready for secondary validation by State offices.
4.1.2 Manual inspection of field records
Prior to data entry to computer an initial inspection of field records is required. This is done in
conjunction with notes received from the observation station on equipment problems and faults,
missing records or exceptional rainfall. Rainfall sheets and charts are inspected for the following:
• Is the station name and code and month and year recorded?
• Does the number of record days correspond with the number of days in the month?
• Are there some missing values or periods for which rainfall has been accumulated during
absence of the observer?
• Have monthly totals of rainfall and rain days been entered?
• Have the autographic rainfall hourly totals been extracted?
• Is the record written clearly and with no ambiguity in digits or decimal points?
Any queries arising from such inspection should be communicated to the observer to confirm
ambiguous data before data entry. Any unresolved problems should be noted and the information
sent forward with the digital data to Divisional/State offices to assist in initial secondary validation.
Any equipment failure or observer problem should be communicated to the supervising field officer
for rectification.
4.1.3 Entry of daily rainfall data
Using the eSWDES module in e-SWIS, the user selects the correct station and daily series. The
screen for entry (or editing) of daily rainfall is displayed, along with the upper warning level used to
flag suspect values (which can be altered for different seasons), and the maximum and minimum
values for that station. For rainfall the minimum value is 0.0 mm, and a rainy day is defined as that
day on which the rainfall is more than 0.0 mm. The user selects the correct year and month, and
enters the daily rainfall value recorded at 08:30 for each date, adding comments where
appropriate. Negative and non-numerical entries are automatically rejected. For each month, the
user also enters the number of rain days, total rainfall and maximum rainfall. The software also
calculates the number of rain days, the cumulative rainfall and the maximum rainfall as the user
enters the data.
Two types of data entry checks are performed for this case of daily rainfall data:
• The number of rain days, total monthly rainfall and maximum rainfall entered by the user are
compared with the values calculated by the software. In the case of a mismatch the user is
25. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 19
prompted by colour highlighting and can refer back to the field documents to see if there was
some error in entering the data. If cumulated values are also available in the field documents,
it becomes quicker to isolate the error.
• The entered daily data are compared against the upper warning level and the maximum limit.
This identifies potentially suspect values to the user who can refer back to the field documents
to see if there was some error in entering the data. If values which exceed the upper warning
level and the maximum limit are actually reported in the field documents, the user should add
an appropriate comment.
Any mismatch remaining after thorough checking of the field documents must be due to incorrect
field computations by the observer and should be communicated to the supervising field officer.
The user should also view entered data graphically to identify potentially suspect data not apparent
in tabular form, which may reflect an error in data entry. There are three ways in which the entered
data can be plotted: daily data for the month, daily data for the year, and monthly totals for the
year.
Missing data When data are missing, the corresponding cell is left as -999 (not zero) and a
comment entered against that day.
Accumulated data Where the observer has missed readings over a period of days and an
accumulated total is subsequently measured, the cells corresponding to the missed days are left
as -999 (not zero) and a comment entered against the date of the accumulation to specify the
period over which the accumulation has occurred (e.g. Accumulated from 23 to 27 Sep). There are
occasions when the observer is legitimately absent from her/his station, for example on account of
sickness. The observer should be encouraged to leave such spaces “Missing” or “Accumulated”
rather than guess the missing values. The completion procedures (Section 4.4), based on
adjoining information, are better able to estimate such missing values.
4.1.4 Entry of rainfall data at twice daily interval
Using the eSWDES module in e-SWIS, the user selects the correct station and twice-daily series.
The screen for entry (or editing) of twice-daily rainfall is displayed, along with the upper warning
level used to flag suspect values (which can be altered for different seasons), and the maximum
and minimum values for that station. For rainfall the minimum value is 0.0 mm, and a rainy day is
defined as that day on which the rainfall is more than 0.0 mm. The user selects the correct year
and month, and enters the twice-daily rainfall values recorded at 17:30 the previous day and 08:30
for each date, adding comments where appropriate. Negative and non-numerical entries are
automatically rejected. For each month, the user also enters the number of rain days, total rainfall
and maximum rainfall. The software also calculates the number of rain days, the total daily rainfall,
the cumulative rainfall and the maximum rainfall as the user enters the data.
Two types of data entry checks are performed for this case of twice-daily rainfall data:
• The number of rain days, total daily rainfall, total monthly rainfall and maximum rainfall entered
by the user are compared with the values calculated by the software. In the case of a
mismatch the user is prompted by colour highlighting and can refer back to the field
documents to see if there was some error in entering the data. If cumulated values are also
available in the field documents, it becomes quicker to isolate the error.
• The entered daily data are compared against the upper warning level and the maximum limit.
This identifies potentially suspect values to the user who can refer back to the field documents
to see if there was some error in entering the data. If values which exceed the upper warning
level and the maximum limit are actually reported in the field documents, the user should add
an appropriate comment.
26. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 20
Any mismatch remaining after thorough checking of the field documents must be due to incorrect
field computations by the observer and should be communicated to the supervising field officer.
The user should also view entered data graphically to identify potentially suspect data not apparent
in tabular form, which may reflect an error in data entry. There are four ways in which the entered
data can be plotted: twice daily data for the month, daily data for the month, daily data for the year,
and monthly totals for the year.
Missing and accumulated data are handled in the same way as for entry of daily rainfall data
(Section 4.1.3).
4.1.5 Entry of hourly rainfall data
Hourly rainfall data are obtained either from the chart records of ARGs or from the digital data of
TBRs. Digital data can also be imported directly, but can undergo entry checks and be viewed
graphically using this option.
Using the eSWDES module in e-SWIS, the user selects the correct station and hourly series. The
screen for entry (or editing) of hourly rainfall is displayed, along with the upper warning level used
to flag suspect values (which can be altered for different seasons), and the maximum and
minimum values for that station. For rainfall the minimum value is 0.0 mm. The user selects the
correct year and month, and enters the hourly rainfall values, with each row corresponding to a
different day and each column to a different time, adding comments where appropriate. The rainfall
value is entered against the time following the hour in which the rainfall occurred e.g. rainfall falling
and recorded from 11:30 to 12:30 is recorded against 12:30. Negative and non-numerical entries
are automatically rejected. For each day, the user enters the daily total. For each month, the user
also enters the columnar total for each hourly period, the number of rain days, total rainfall and
maximum rainfall. The software also calculates the daily and hourly totals, the number of rain
days, the cumulative rainfall and the maximum rainfall as the user enters the data.
Two types of data entry checks are performed for this case of hourly rainfall data:
• The number of rain days, columnar total for each hourly period, total daily rainfall, total monthly
rainfall and maximum rainfall entered by the user are compared with the values calculated by
the software. In the case of a mismatch the user is prompted by colour highlighting and can
refer back to the field documents to see if there was some error in entering the data. If
cumulated values are also available in the field documents, it becomes quicker to isolate the
error.
• The entered hourly data are compared against the upper warning level and the maximum limit.
This identifies potentially suspect values to the user who can refer back to the field documents
to see if there was some error in entering the data. If values which exceed the upper warning
level and the maximum limit are actually reported in the field documents, the user should add
an appropriate comment.
Any mismatch remaining after thorough checking of the field documents must be due to incorrect
field computations by the observer and should be communicated to the supervising field officer.
The user should also view entered hourly data for the day and month graphically to identify
potentially suspect data not apparent in tabular form, which may reflect an error in data entry.
Missing and accumulated data are handled in the same way as for entry of daily rainfall data
(Section 4.1.3).
27. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 21
4.1.6 Import/entry of digital data
Digital data from TBRs can take two forms: time mode where the number of tips in a pre-set time
interval (e.g. 1 hour, 15 minutes, etc) are recorded, or in event mode where the times of every tip
are recorded, thereby producing a more flexible record for subsequent analysis. TBR data can be
imported directly should an appropriate import interface be available (bespoke to each type of data
logger), and can undergo entry checks and be viewed graphically as described in Section 4.1.5.
Time mode data are imported to an appropriate equidistant time series, whilst event mode data are
imported to a non-equidistant time series.
4.2 Primary validation
4.2.1 Overview
Primary validation is primarily done at a Sub-Divisional office level where staff are in close contact
to field staff who have made the observations and/or collected the chart or digital data. Primary
validation is carried out using e-SWIS, the data entry module of which replicates the SWDES
software from HPI, and is referred to as eSWDES. Primary validation (see Table 3.1) of field data
from observational stations is required to be completed at Sub-Divisional office level by the 10th
working day of the following month (e.g. for June data by 10th
working day in July), ready for
secondary validation by State offices. This time schedule ensures that any obvious problems (e.g.
indicating an instrument malfunction, observer error, etc) are spotted at the earliest opportunity and
resolved. Other problems may not become apparent until more data have been collected, and
data can be viewed in a longer-term context during secondary validation.
Primary validation of rainfall data focuses on validation within a single data series by making
comparisons between individual observations and pre-set physical limits, and between two
measurements of rainfall at a single station (e.g. daily rainfall from an SRG and accumulated daily
rainfall from an ARG/TBR). The high spatial and temporal variability of rainfall data compared to
other climate variables makes validation of rainfall more difficult. This is particularly the case on
the Indian sub-continent, experiencing a monsoon-type climate involving convective precipitation.
Examples of many of the techniques described in this section are given in Hydro-Meteorology
Training Module 08 “How to carry out primary validation for rainfall data” and Training Module 10
“How to correct and complete rainfall data”.
4.2.2 Typical errors
Staff should be aware of typical errors in rainfall measurement, listed in Table 4.1, and these
should be considered when interpreting data and possible discrepancies (SW8-OM(I) Chapter 5.2).
SRG errors from most of these sources are very difficult to detect from the single record of the
standard raingauge, unless there has been a gross error in reading or transcribing the value
(Section 4.2.3). Errors are more readily detected if there is a concurrent record from an ARG or
TBR. As these too are subject to errors (of a different type), comparisons with the SRG are very
important (Section 4.2.4). The final check by comparison with raingauges at neighbouring stations
should show up further anomalies, especially for those stations which do not have an ARG or TBR
at the site. This is carried out during secondary validation where more gauges are available for
comparison (Section 4.3).
4.2.3 Comparison with upper warning level and maximum and minimum limits
Both hourly and daily rainfall data should be validated against physical limits, which are required to
be quite wide to avoid the possibility of rejecting true extreme values. For rainfall data, the
minimum limit is 0.0 mm. The maximum limit will vary spatially over India with climatic region and
28. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 22
Table 4.1 Measurement errors for rainfall data
SRG measurement errors
• Observer reads measuring glass incorrectly
• Observer enters amount incorrectly in the field sheet
• Observer reads gauge at the wrong time (i.e. the correct amount may thus be allocated to the wrong
day)
• Observer enters amount to the wrong day
• Observer uses wrong measuring glass (i.e., 200 cm
2
glass for 100 cm
2
gauge, giving half the true
rainfall, or 100 cm
2
glass for 200 cm
2
gauge giving twice the true rainfall
• Observed total exceeds the capacity of the gauge
• Instrument fault - gauge rim damaged so that collection area is affected
• Instrument fault - blockage in raingauge funnel so that water does not reach collection bottle and may
overflow or be affected by evaporation
• Instrument fault - damaged or broken collector bottle and leakage from gauge
ARG measurement errors
• Potential measurement faults are primarily instrumental rather than caused by the observer
• Funnel is blocked or partly blocked so that water enters the float chamber at a different rate from the rate
of rainfall
• Float is imperfectly adjusted so that it syphons at a rainfall volume different from 10 mm
• In very heavy rainfall the float rises and syphons so frequently that individual pen traces cannot be
distinguished
• Clock stops; rainfall not recorded or clock is either slow or fast and thus timings are incorrect
• Float sticks in float chamber; rainfall not recorded or recorded incorrectly
• Observer extracts information incorrectly from the pen trace
TBR measurement errors
• Funnel is blocked or partly blocked so that water enters the tipping buckets at a different rate from the
rate of rainfall
• Buckets are damaged or out of balance so that they do not record their specified tip volume
• Reed switch fails to register tips
• Reed switch double registers rainfall tips as bucket bounces after tip. (better equipment includes a
debounce filter to eliminate double registration)
• Failure of electronics due to lightning strike etc. (though lightning protection usually provided)
• Incorrect set up of measurement parameters by the observer or field supervisor
orography, and maximum limits for 1-hour and 1-day rainfall should be based of analysis of historic
data for the station or IMD maps of 1-hour maximum rainfall and 1-day maximum rainfall. However,
validation of rainfall data against a maximum value does not discriminate those comparatively
frequently occurring erroneous data which are less than the prescribed maximum limit. In view of
this, it is advantageous to consider an upper warning level, which can be employed to filter high
data values which are not expected to occur frequently. For daily rainfall data, this limit can be set
statistically e.g. to 99th
percentile of actual rainfall values excluding zero values. A similar statistic
can be employed for obtaining a suitable upper warning level for hourly rainfall data. Setting such
warning levels and limits for each station ensures rapid filtering of potentially erroneous values,
and should result is not too many and not too few, data values being flagged as suspect.
During data entry, hourly and daily data are compared against the upper warning level and the
maximum limit. If values which exceed the upper warning level and the maximum limit are actually
reported in the field documents, the user should have considered the values suspect and added an
appropriate comment, for further attention during primary validation. Data entries which are more
than the prescribed upper warning level or the maximum limit may imply that the earlier maximum
has indeed been crossed. In such cases, it is expected that there will be a few nearby stations
recording similar higher rainfall to support such inferences, and this should be reviewed during
secondary validation.
29. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 23
4.2.4 Comparison of daily and sub-daily rainfall data
For stations with an ARG or a TBR, an SRG is always also available. Thus, daily rainfall data are
available from two independent sources. The accumulated daily rainfall from the sub-daily
raingauge should be compared with the daily rainfall at the daily raingauge, to check for
consistency. Differences which are less than 5% can be attributed to exposure, instrument
accuracy and precision in tabulating the analogue records and are ignored, but differences greater
than 5% may indicate potential errors and should be examined further. The observation made
using the SRG is traditionally regarded as comparatively more reliable. This is based on the
assumption that there is higher degree of possibility of malfunctioning of autographic or digital
recorders owing to their mechanical and electronic systems. However, significant systematic or
random errors are also possible in the SRG as shown in Table 4.1. If the error is in the
autographic or digital records, then it should be possible to relate it either to instrumental or
observational errors. Moreover, such errors tend to repeat under similar circumstances.
Comparison of daily and accumulated sub-daily rainfall data may be carried out in tabular or
graphical form, with an additional table column for those days where the discrepancy is more than
5%, or a second graph axis showing percent discrepancy:
• Where the recording gauge gives a consistently higher or lower total than the daily gauge, then
the recording gauge could be out of calibration and either tipping buckets (TBR) or floats (ARG)
need recalibration → Accept SRG and adjust ARG or TBR
• Where agreement is generally good but difference increases in high intensity rainfall suggests
that for the ARG either (i) the syphon is working imperfectly in high rainfall or (ii) the chart trace
is too close to distinguish each 10 mm trace (underestimate by multiples of 10 mm), and for the
TBR (i) the gauge is affected by bounce sometimes giving double tips → Accept SRG and
adjust ARG or TBRG
• Where a day of positive discrepancy is followed by a negative discrepancy and rainfall at the
recording gauge was occurring at the observation hour, then it is probable that the observer
read the SRG at a different time from the ARG. The sum of SRG readings for successive days
should equal the 2-day total for the ARG or TBR → Accept ARG or TBR and adjust SRG
• Where the agreement is generally good but isolated days have significant differences, then the
entered hourly data should be checked against the field documents. Entries resulting from
incorrect entry are corrected. Check that water added to a TBR for calibration is not included in
rainfall total. Otherwise there is probable error in the SRG observation → Accept ARG or TBR
and adjust SRG
• Where the values reported for the daily rainfall by the SRG and ARG/TBR correspond exactly
for considerable periods, it is conceivable that the observer has forcefully manipulated one or
both datasets. Variation must exist due to variance in the catch and instrument and
observation variations. Both hourly and daily data should be checked against the field
documents to attempt to ascertain which has been manipulated.
In the case where the SRG record is accepted records from autographic gauges at neighbouring
stations can be used in conjunction with the SRG at the station to correct the ARG/TBR record at
the station. This involves hourly distribution of the daily total from the SRG at the station by
reference to the hourly distribution at one or more neighbouring stations. Donor (or base) stations
are selected by making comparison of cumulative plots of events in which autographic records are
available at both stations and selecting the best available for estimation. If the daily rainfall on the
day in question at the station under consideration is Dtest and the hourly rainfall for the same daily
period at the selected neighbouring donor station are Hbase,i (i = 1, 24), the hourly rainfall at the
station under consideration, Htest,i are obtained as:
30. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 24
The procedure may be repeated for more than one neighbouring donor station and the average or
resulting hourly totals calculated.
In the case where the ARG/TBR record is accepted it can be used to correct the SRG record at
the station. The SRG data values are made equal to the accumulated daily or twice-daily totals
from the ARG/TBR.
Where a shift in one of the data series has been detected, this can be removed after looking at the
field documents and corresponding tabulated data and adjusting the data appropriately. The
adjusted data should be flagged as corrected and an appropriate comment added.
Where a doubtful or incorrect rainfall value is identified, and there is any uncertainty as to the
correct action, this should be marked with an appropriate flag to indicate that it is suspect. The
data flagged as suspect are reviewed at the time of secondary validation.
4.3 Secondary validation
4.3.1 Overview
Secondary validation of rainfall data is primarily carried out at State DPCs, to take advantage of the
information available from a larger area. Secondary validation is carried out using e-SWIS, the
validation module of which replicates the HYMOS software from HPI, and is referred to as
eHYMOS. Data may also be exported to Excel for secondary validation. For the Hydrology
Project, secondary validation (see Table 3.1) done at State level should be completed by the end
of the following month (e.g. for June data by 31st
July). Some secondary validation (including
comparison with IMD data) will not be possible until the end of the hydrological year when the
entire year’s data can be reviewed in a long-term context, so data should be regarded as
provisional approved data until then (e.g. for June data by the end of the hydrological year plus 3
months), after which data should be formally approved and made available for dissemination to
external users.
Data entering secondary validation have already received primary validation on the basis of
knowledge of the station and instrumentation and field documents. Data may have been flagged
as missing, accumulated, shifted or suspect for some other reason e.g. a mismatch in the number
of rain days. Secondary validation focuses on comparisons with neighbouring stations to identify
suspect values. However, data processing staff should continue to be aware of field practice and
instrumentation and the associated errors which can arise in data. Some of the secondary
validation checks are oriented towards the specific types of errors just mentioned, whilst others are
general in nature and lead to identification of spatial inconsistencies in the data. Rainfall poses
particular problems for spatial comparisons because of the limited or uneven correlation between
stations. When rainfall is convectional in type, it may rain heavily at one location, whilst another
only a few miles away may remain dry. Over a month or a monsoon season, such spatial
unevenness tends to be smoothed out and aggregated totals are much more closely correlated.
Spatial correlation in rainfall depends on type of precipitation, physiographic characteristics of the
region, duration (decreases as duration decreases) and distance (decreases as distance
increases). Examples of many of the techniques described in this section are given in Hydro-
Meteorology Training Module 09 “How to carry out secondary validation of rainfall data” and
Training Module 10 “How to correct and complete rainfall data”.
31. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 25
4.3.2 Comparison with data limits for totals at longer durations
Daily rainfall data are aggregated to monthly, seasonal and yearly time intervals for checking if the
resulting data series is consistent with the prescribed data limits for such time intervals. This is a
useful technique for identifying small systematic or persistent observation errors in rainfall that are
not apparent at a daily time interval but which tend to accumulate with time and, therefore, become
more visible. As well as upper warning levels or maximum limits, for monsoon months and yearly
values, use of lower warning levels can be made to see if certain values are unexpectedly low and
thus warrant a closer look. This is a valuable secondary check for SRGs at which there is no sub-
daily raingauge for comparison. Suspect aggregated values are flagged and commented
appropriately for further validation on the basis of data from nearby stations.
4.3.3 Comparison of rainfall from multiple stations
A combination of graphical and tabular displays of hourly, daily, monthly, seasonal and annual
rainfall data from multiple stations in a region provides an efficient way of identifying anomalies.
Where only two stations are involved in the comparison, the identification of an anomaly does not
necessarily indicate which station is at fault. Specific checks to make include:
• Do the daily blocks of rain days generally coincide in start day and finish day?
• Are there exceptions that are misplaced, starting one day early or late?
• Is there a consistent pattern of misfit for a station through the month?
• Are there days with no rainfall at a station when (heavy) rainfall has occurred at all
neighbouring stations?
For multiple rainfall time series plots, select a set of stations within a small area with an expectation
of spatial correlation including in the set, if possible, one or more stations which historically have
been more reliable. Plot the rainfall series as histograms, preferably in different colours for each
station. Side by side plotting of histograms permits comparison on the magnitudes of rainfall at
different stations, whilst one above the other plotting makes time shifts easier to detect.
This is a valuable secondary check for SRGs at which there is no sub-daily raingauge for
comparison. Rainfall occurs in dry and wet spells and SRG-only observers may fail to record the
zeros during the dry spells and, hence, lose track of the date when the next rain arrives. When
ancillary climate data are available, this may be used to compare with rainfall data e.g. a day with
unbroken sunshine in which rain has been reported suggests that rainfall has been reported for the
wrong day. However, most comparisons are not so clear cut and the user should be aware that
there are a number of possibilities:
• Rainfall data only on the wrong day - anomalies between rainfall and climate and between
rainfall and neighbouring rainfall
• Rainfall and climate data both reported on the wrong day - hence no anomaly between them
but discrepancy with neighbouring stations
• Rainfall and climate both reported on the correct day - the anomaly was in the occurrence of
rainfall e.g. no rainfall at one site but at neighbouring sites. In this case, climate variables are
likely to have been shared between neighbouring stations even if rainfall did not occur
Unexplained anomalies should initially be followed up by checking the field documents to check for
unnoticed mistakes during data entry or primary validation, in which case the data can be corrected
accordingly. If necessary, the anomaly should be communicated to the supervising field officer
and observer to confirm data and/or rectify problems. Data still regarded as suspect after follow-up
checking are flagged and commented appropriately for further validation on the basis of data from
nearby stations.
32. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 26
Figure 4.1 Definition of test and neighbouring stations
4.3.4 Spatial homogeneity testing of rainfall
Spatial homogeneity testing of rainfall data is a technique where the observed rainfall at a station
station is compared with an estimate of the rainfall at that station, based on the weighted average
of rainfall from nearby stations. This approach depends on the degree of spatial consistency of the
rainfall, which is primarily based on the actual spatial correlation. This check for spatial
consistency can be carried out for various durations of rainfall accumulations. Wherever the
difference between the observed and the estimated values exceeds the expected limiting value,
such observed values are considered suspect values and flagged for further investigation to
ascertain the possible causes of the differences. It is only possible to include a brief description of
the technique below, but more detailed information is provided in SW8-OM(II) Chapter 2.6 and
Chapter 3.6.
Firstly, the nearby stations to be used should be chosen. The stations selected as neighbours
should be physically representative of the area in which the “test station” (the station under
scrutiny) is situated. The following criteria are used to select the neighbouring stations (Figure
4.1):
• The distance between the test and the neighbouring station should be less than a specified
maximum correlation distance, say Rmax km
• A maximum of 8 neighbouring stations can be considered for interpolation
• To reduce the spatial bias in selection, it is appropriate to consider a maximum of only two
stations within each quadrant.
Secondly, having selected the neighbouring stations, the estimation of the spatially interpolated
rainfall is made at the test station, by computing the weighted average of the rainfall observed at
neighbouring stations. The estimate of the interpolated value at the test station based on the
observations at M neighbouring stations is given as:
Where: Pest,j = estimated rainfall at the test station at time j
Pi,j = observed rainfall at the neighbour station i at time j
33. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 27
Di = distance between the test and the neighbouring station i
Mbase = number of neighbouring stations taken into account
b = power of distance D used for weighting rainfall values at individual station e.g. 2
Thirdly, this estimated rainfall is compared with the observed rainfall at the test station and the
difference is considered insignificant if:
• The difference between the observed and estimated rainfall at the test station is less than or
equal to the admissible absolute difference Xabs
• The difference between the observed and estimated rainfall at the test station is less than or
equal to a multiplier Xrel of the standard deviation SPest,j of the neighbouring values given by:
Where differences between the observed and estimated are unacceptably high, the observed
value should be flagged “+” or “-”, depending on whether the observed rainfall is greater or less
than the estimated one. Typical rainfall measurement errors show up with specific patterns of “+”
and “-”.
The limits Xabs and Xrel are chosen by the user and have to be based on the spatial variability of
rainfall, and may be altered seasonally. They are normally determined on the basis of experience
with the historical data with the objective of flagging a few values (say 2-3%) as suspect values. It
is customary to select a reasonably high value of Xabs and a value of Xrel which avoid having to
deal with a large number of difference values in the lower range where differences are more likely
to occur and have less effect on the overall rainfall total.
4.3.5 Double mass analysis
Double mass analysis is a technique to detect a systematic shift, like abrupt or gradual changes, in
a rainfall time series, persisting for a considerable period of time. Such inconsistencies can occur
for various reasons:
• The raingauge might have been installed at different sites in the past
• The exposure conditions of the gauge may have undergone a significant change due to the
growth of trees or construction of buildings in its proximity
• There might have been a change in the instrument, say from 125 mm to 200 mm raingauge
• The raingauge may have been faulty for a considerable period etc
A note may be available in the station files of the known changes of site and instruments and can
be used to corroborate the detection of inconsistencies. The application of double mass analysis
to rainfall data is not be possible until a significant amount of historical data is available. The
accumulated rainfall at the test station (the station under scrutiny) is compared with another
accumulated rainfall series that is expected to be homogeneous. The homogeneous time series
can be from an individual reliable station or be an average of reliable time series from neighbouring
stations, referred to as the base station(s). Accumulation of rainfall can be made from daily data to
monthly or yearly duration. It is only possible to include a brief description of the technique below,
but more detailed information is provided in SW8-OM(II) Chapter 2.12 and Chapter 3.5.
Firstly, the double mass plot between the accumulated rainfall values in absolute or percent form at
test and base stations is drawn and observed for any visible change in its slope. The tabular
output giving the ratio between the accumulated rainfall values at test and base stations in
absolute and percent form is also obtained. The analysis can be carried for only a part of the years
34. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 28
or months, if there are some missing rainfall values within the time series (which may themselves
indicate some change at the station).
Secondly, any visible change in the slope of the double mass plot should be noted. If the data of
the test station is homogeneous and consistent with the data of the base station(s), the double
mass curve will show a straight line. An abrupt change in the test rainfall series will create a break
in the double mass curve, whereas a trend will create a curve. A change in slope is not usually
considered significant unless it persists for at least 5 years, and it does not imply that either period
is incorrect, simply that it is inconsistent. Furthermore, double mass analysis is based on the
assumption that only a part of the rainfall time series under consideration is subject to systematic
shift. Where the whole rainfall time series has such a shift, the double mass analysis will fail to
detect any inconsistency. Any significant inconsistency that is detected should be investigated
further to explore possible causes. If the inconsistency is caused by changed exposure conditions
or shift in the station location or systematic instrumental error, the rainfall series should be
considered suspect.
Thirdly, the suspect part of the rainfall series can be made homogeneous by suitably transforming
it before/after the period of shift as required. The earlier part of the record may be adjusted so that
that the entire record is consistent with the present and continuing record, or the latter part of the
record may be adjusted when the source of the inconsistency is known and has been removed or
where the record has been discontinued. Transformation is carried out by multiplying it by a
correction factor which is the ratio of the slope of the adjusted mass curve to the slope of the
unadjusted mass curve.
In the double mass plot shown in Figure 4.2, there is a distinct break at time T1. If the start and
end times of the period under consideration are T0 and T2, respectively, then the slopes of the
curve before α1 and after α2 the break point can be expressed as:
Figure 4.2 Definition sketch for double mass analysis
35. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 29
In the case that the earlier part between T0 and T1 needs to be corrected for, the correction factor
and the corrected observations at the test station can be expressed as:
After making such a correction, the double mass curve should be plotted again to check that there
is no significant change in the slope of the curve.
4.4 Correction and completion
4.4.1 Overview
Completion – the processing of filling in missing values and correcting erroneous values – is done
as a continuous process with primary and secondary validation. Although the HIS Manual SW
separates correction and completion in SW8-OM(II) Chapter 3 from secondary validation in SW8-
OM(II) Chapter 2, and from primary validation in SW8-OM(I) Chapter 5, there is substantial overlap
between the techniques used. In this Handbook, some correction and completion techniques have
been included in the appropriate parts of Sections 4.2 and 4.3, and others are described below.
Examples of many of the techniques described, which should be carried out by experienced staff
with appropriate training, are given in Hydro-Meteorology Training Module 10 “How to correct and
complete rainfall data”.
The majority of secondary validation, and therefore the majority of correction and completion, is
carried out by State DPCs to take advantage of the information available from a larger area. For
the Hydrology Project, correction and completion (see Table 3.1) should be completed by the end
of the following month (e.g. for June data by 31st
July). Some secondary validation, correction and
completion will not be possible until the end of the hydrological year when the entire year’s data
can be reviewed in a long-term context, so data should be regarded as provisional approved data
until then (e.g. for June data by the end of the hydrological year plus 3 months), after which data
should be formally approved and made available for dissemination to external users.
Procedures for correction and completion depend on the type of error and the availability of
suitable source records with which to estimate. It should be recognised that values estimated from
other gauges are inherently less reliable than values properly measured. There will be
circumstances where no suitable neighbouring observations or stations are available, such that
missing values should be left as -999 and incorrect values should be set to -999, and suspect
original values should be given the benefit of the doubt and retained in the record with an
appropriate flag.
4.4.2 Correcting missing and erroneous data
Missing values (-999 or incorrect zeros) may be the result of the observer failing to make an
observation, failing to enter the observation in the record sheet, or entering the observation
incorrectly. For rainy periods, missed values will be anomalous in the multiple station tabulation
and plot and will be indicated by a series of “-” departures in the spatial homogeneity test.
Where such missed entries are confidently identified, the missed values should be replaced by the
estimates derived from neighbouring stations by spatial interpolation. This is also the approach for
values confidently identified as erroneous. Where there is some doubt as to the interpretation, the
36. Hydrological Information System May 2014
HP II
Last Updated: 19/05/2014 05:01
Filename: MET Handbook.docx
Page 30
value should be left unchanged but flagged as suspect. There are three analytical procedures for
estimating rainfall by spatial interpolation:
• Arithmetic average – applied if the average annual rainfall of the station under consideration
is within 10% of the average annual rainfall at the neighbouring stations. The missing or
erroneous rainfall at the station under consideration is estimated as the simple average of
neighbouring stations. Thus, if the estimate for the missing or erroneous rainfall at the station
under consideration is Ptest and the rainfall at M adjoining stations is Pbase,i (i = 1 to M), then:
Usually, averaging of three or more neighbouring stations is considered to give a satisfactory
estimate.
• Normal ratio – applied if the average annual rainfall of the station under consideration differs
from the average annual rainfall at the neighbouring stations by more than 10%. The
erroneous or missing rainfall at the station under consideration is estimated as the weighted
average of the data at the neighbouring stations. The rainfall at each of the neighbouring
stations is weighted by the ratio of the average annual rainfall at the station under
consideration and average annual rainfall of the neighbouring station. The rainfall for the
missing or erroneous value at the station under consideration is estimated as:
Where: Ntest = annual average rainfall at the station under consideration
Nbase,i = annual average rainfall at the adjoining stations (for i = 1 to M)
A minimum of three neighbouring stations should generally be used for obtaining good
estimates using the normal ratio method.
• Distance power – this is the approach used for the spatial homogeneity test in Section 4.3.4 of
this Handbook, and weights neighbouring stations on the basis of their distance from the
station under consideration, on the assumption that closer stations are better correlated than
those further away and that, beyond a certain distance, they are insufficiently correlated to be
of use. Spatial interpolation is made by weighing the adjoining station rainfall as inversely
proportional to some power (e.g. 2) of the distances from the station under consideration.
However, due to prevailing wind conditions or orographic effects, spatial heterogeneity may be
present in which case, normalised values rather than actual values should be used in the
interpolation. The observed rainfall values at the neighbouring stations are multiplied by the
ratio of the average annual rainfall at the test station and the average annual rainfall at the
neighbouring stations:
Where: Pcorr,i,j = for corrected rainfall value at the neighbouring station i at time j
Ntest = annual average rainfall at the station under consideration
Nbase,i = annual average rainfall at the neighbouring stations (for i = 1 to Mbase)
4.4.3 Correcting accumulated data
Should an observer miss one or more readings, he/she may make one of three choices for the
missed period of record: