Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Final met handbook 180514

763 views

Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Final met handbook 180514

  1. 1. HP IIIndian Hydrology Project Technical Assistance (Implementation Support) and Management Consultancy Hydro-Meteorology Handbook: Precipitation and Climate May 2014
  2. 2. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Hydro-Meteorology Handbook: Precipitation and Climate Issue and Revision Record Revision Date Originator Checker Approver Description 0 21/05/14 Helen Houghton-Carr Version for approval 1 2 3
  3. 3. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page i Contents Contents i Glossary iii 1. Introduction 1.1 HIS Manual 1.2 Other HPI documentation 1.3 Rainfall data in groundwater studies 1 2 3 4 2. The Data Management Lifecycle in HPII 5 2.1 Use of hydrological information in policy and decision- making 2.2 Hydrological monitoring network design and development 2.3 Data sensing and recording 2.4 Data validation and archival storage 2.5 Data synthesis and analysis 2.6 Data dissemination and publication 2.7 Real-time data 5 6 6 6 7 8 8 3. Hydro-Meteorological Monitoring Stations and Data 10 3.1 Types of hydro-meteorological monitoring station 3.2 Hydro-meteorological monitoring networks 3.3 Site inspections, audits and maintenance 3.4 Data sensing and recording 3.5 Data processing 10 10 14 14 15 4. Rainfall Data Processing and Analysis 18 4.1 Data entry 4.2 Primary validation 4.3 Secondary validation 4.4 Correction and completion 4.5 Compilation 4.6 Analysis 18 21 24 29 33 37 5. Snow Data Processing and Analysis 40 5.1 Snow data in the Hydrology Project 5.2 Data entry 5.3 Primary validation 5.4 Secondary validation 5.5 Analysis 40 41 43 43 44 6. Climate Data Processing and Analysis 46 6.1 Data entry 6.2 Primary validation 6.3 Secondary validation 6.4 Correction and completion 6.5 Analysis 46 49 51 54 56 7. Data Dissemination and Publication 59 7.1 Hydro-meteorological products 7.2 Annual reports 7.3 Periodic reports 7.4 Special reports 59 59 62 63
  4. 4. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page ii 7.5 Dissemination to hydrological data users 63 References 64 Annex I States and agencies participating in the Hydrology Project 65 Annex II Summary of distribution of hard copy of HPI HIS Manual Surface Water 66 Annex III Summary of distribution of hard copy of HPI HIS Manual Groundwater 67 List of figures 1.1 Hydrometric information lifecycle 1 4.1 Definition of test and neighbouring stations 26 4.2 Definition sketch for double mass analysis 28 4.3 Example of basin area divided into Theissen polygons 35 4.4 Example of drawing isohyets using linear interpolation 35 List of tables 1.1 HPI hydro-meteorology training modules 4 2.1 Hydro-meteorological data processing timetable for data for month n 8 3.1 Where to go in the HIS Manual SW for hydro-meteorological data management guidance: rainfall and snow 11 3.1 cont/ Where to go in the HIS Manual SW for hydro-meteorological data management guidance: climate and evaporation 12 4.1 Measurement errors for rainfall data 22 5.1 Measurement errors for snow data 44 6.1 Measurement errors for climate data 50
  5. 5. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page iii Glossary ADCP Acoustic Doppler Current Profiler ARG Autographic Rain Gauge AWS Automatic Weather Station BBMB Bhakra-Beas Management Board CGWB Central Ground Water Board CPCB Central Pollution Control Board CWC Central Water Commission CWPRS Central Water and Power Research Station Div Division DPC Data Processing Centre DSC Data Storage Centre DWLR Digital Water Level Recorder e-GEMS Web-based Groundwater Estimation and Management System (HPII) eHYMOS Web-based Hydrological Modelling System (HPII) eSWDES Web-based Surface Water Date Entry System in e-SWIS (HPII) e-SWIS Web-based Surface Water Information System (HPII) FCS Full Climate Station GEMS Groundwater Estimation and Management System (HPI) GW Groundwater GWDES Ground Water Data Entry System (HPI) GWIS Groundwater Information System (GPI) HDUG Hydrological Data User Group HIS Hydrological Information System HP Hydrology project (HPI Phase I, HPII Phase II) HYMOS Hydrological Modelling System (HPI) IMD India Meteorological Department Lab Laboratory MoWR Ministry of Water Resources NIH National Institute of Hydrology SRG Standard Rain Gauge Stat Station Sub-Div Sub-Division SW Surface Water SWDES Surface Water Data Entry System (HPI) TBR Tipping Bucket Raingauge ToR Terms of Reference WISDOM Water Information System Data Online Management (HPI) WQ Water Quality
  6. 6. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page iv
  7. 7. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 1 1. Introduction This Hydrology Project Phase II (HPII) Handbook provides guidance for the management of hydro- meteorological data on rainfall, snow and other climate variables. The data are managed within a Hydrological Information System (HIS) that provides information on the spatial and temporal characteristics of the quantity and quality of surface water, including hydro-meteorology, and groundwater. The information is tuned to the requirements of the policy makers, designers and researchers to provide evidence to inform decisions on long-term planning, design and management of water resources and water use systems, and for related research activities. The Indian States and Central Agencies participating in the Hydrology Project are listed in Annex I. However, this Handbook is also relevant to non-HP States. It is important to recognise that there are two separate issues involved in managing hydro- meteorological information. The first issue covers the general principles of understanding monitoring networks, of collecting, validating and archiving data, and of analysing, disseminating and publishing data. The second covers how to actually do these activities using the database systems and software available. Whilst these two issues are undeniably linked, it is the first – the general principles of data management - that is the primary concern. This is because improved data management practices will serve to raise the profile of Central/State hydrometric agencies in government and in the user community, highlight the importance of hydro-meteorological data for the design of water-related schemes and for water resource planning and management, and motivate staff, both those collecting the data and those in data centres. This Handbook aims to help HIS users locate and understand documents relevant to hydro- meteorology in the library available through the Manuals page on the Hydrology Project website. The Handbook is a companion to the HIS manuals. The Handbook makes reference to the six stages in the hydrometric information lifecycle (Figure 1.1), in which the different processes of data sensing, manipulation and use are stages in the development and flow of information. The cycle and associated HIS protocols are explored more fully in Section 2. Subsequent sections cover different stages of the cycle for different hydro-meteorological variables. Figure 1.1 Hydrometric information lifecycle (after: Marsh, 2002)
  8. 8. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 2 1.1 HIS Manual The primary reference sources are the HIS Manual Surface Water (SW) and HIS Manual Groundwater (GW), two of many hundreds of documents generated during Hydrology Project Phase I (HPI) to assist staff working in observation networks, laboratories, data processing centres and data communication systems to collect, store, process and disseminate hydrometric data and related information. During HPI, special attention was paid to the standardisation of procedures for the observation of variables and the validation of information, so that it was of acceptable quality and compatible between different agencies and States, and to facilities for the proper storage, archival and dissemination of data for the system, so that it was sustainable in the long-term. Therefore, the majority of the documents produced under HPI, particularly those relating to fundamental principles, remain valid through and beyond HPII. Some parts of the guides, manuals and training material relating to HPI software systems (SWDES, HYMOS, WISDOM, GWDES, GEMS, GWIS) have been partially or wholly superseded as replacement Phase II systems (e- GEMS, e-SWIS) become active. The HIS Manual SW and HIS Manual GW describe the procedures to be used to arrive at a sound operation of the HIS in regard to rainfall, snow and climate data. The HIS Manual SW and HIS Manual GW each consist of 10 volumes. Each volume contains one or more of the following manuals, depending on the topic: • Design Manual (DM) - procedures for the design activities to be carried out for the implementation and further development of the HIS. • Field Manual (FM) or Operation Manual (OM) – detailed instructions describing the activities to be carried out in the field (station operation, maintenance and calibration), at the laboratory (analysis), and at the Data Processing Centres (data entry, validation, processing, dissemination, etc). Each Field/Operation Manual is divided into a number of parts, where each part describes a distinct activity at a particular field station, laboratory or data processing centre. • Reference Manual (RM) - additional or background information on topics dealt with or deliberately omitted in the Design, Field and Operation Manuals. Those HIS Manual SW/GW volumes relevant to rainfall and climate are: SW/GW Volume 1: Hydrological Information System: a general introduction to the HIS, its structure, HIS job descriptions, Hydrological Data User Group (HDUG) organisation and user data needs assessment. The content of the SW and GW volumes is identical. • Design Manual • Field Manual  Part II: Terms of Reference for HDUG  Part III: Data needs assessment SW/GW Volume 2: Sampling Principles: units, principles of sampling in time and space and sampling error theory. The content of the SW and GW volumes is identical. • Design Manual SW/GW Volume 3: Hydro-meteorology: network design, implementation, operation and maintenance. The content of the SW and GW volumes is identical. • Design Manual • Field Manual  Part I: Network design and site selection  Part II: Standard raingauge station (SRG) operation and maintenance
  9. 9. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 3  Part III: Autographic raingauge station (ARG or TBR (and SRG)) operation and maintenance  Part IV: Full climate station (FCS) operation and maintenance  Part V: Field inspections, audits maintenance and calibration • Reference Manual SW Volume 8: Data processing and analysis: specification of procedures for Data Processing Centres (DPCs). • Operation Manual  Part I: Data entry and primary validation  Part II: Secondary validation  Part III: Final processing and analysis  Part IV: Data management GW Volume 8: Data processing and analysis • Operation Manual  Part V: Groundwater Year Book SW Volume 10: Surface Water protocols: outline of protocols for data collection, entry, validation and processing, communication, inter-agency validation, data storage and dissemination, HIS training and management. • Operation Manual  Data entry forms In this Handbook, individual parts of the HIS Manual SW/GW are referred to according to the nomenclature “SW/GWvolume-manual(part)” e.g. SW Volume 3: “Hydro-meteorology” Field Manual Part II: “Standard raingauge station (SRG) operation and maintenance” is referred to as SW3-FM(II), and GW Volume 8: “Data processing and analysis” Operation Manual Part V: “Groundwater Year Book” is referred to as GW8-OM(V). A hard copy of the relevant manuals should be available for the locations listed in Annex II. For example, a hard copy of SW3-FM(II) should be available at all meteorological stations where rainfall measurement with an SRG takes place. Similarly, SW8-OM(I) should be available at all Data Processing Centres where data entry and primary validation take place. As noted, there is some inevitable overlap and repetition between the HIS Manual SW and the HIS Manual GW (e.g. Volume 3). In the following sections of this Handbook, reference is generally made only to the HIS Manual SW, as the majority of hydro-meteorological reference material is incorporated in here, unless there is important additional information in the HIS Manual GW. 1.2 Other HPI documentation Other HPI documents of relevance to hydro-meteorology include: • The e-SWIS software manual, and the SWDES and HYMOS software manuals - although SWDES and HYMOS are being superseded by e-SWIS in HPII, to promote continuity, e-SWIS contains eSWDES and eHYMOS modules. • “Illustrations: hydrological observations” – an illustrative booklet demonstrating how to make measurements of rainfall, water level and flow at stations, and also how to carry out an inspection at those stations.
  10. 10. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 4 Table 1.1 HPI hydro-meteorology training modules Topic Module Title Meteorology 07 How to make data entry for rainfall data 08 How to carry out primary validation for rainfall data 09 How to carry out secondary validation of rainfall 10 How to correct and complete rainfall data 11 How to compile rainfall data 12 How to analyse rainfall data 13 How to report on rainfall data 14 MISSING – How to process evaporation data 15 How to make data entry for climatic data 16 How to carry out primary validation for climatic data 17 How to carry out secondary validation of climatic data 18 MISSING – How to correct and complete climatic data 19 How to analyse climatic data 20 How to report on climatic data Hydrometry 43* Statistical Analysis with Reference to Rainfall & Discharge Data 44* How to carry out correlation and spectral analysis 45* How to review Monitoring Networks * Hydrometry modules also relevant to Hydro-Meteorology. 43 and 44 present statistical analysis techniques as applied to, say, analyse rainfall data. • “Surface Water O&M norms” – a maintenance guide for hydro-meteorology, stage-discharge and water quality instrumentation and equipment. • “Surface Water Yearbook” – a template for a Surface Water Yearbook published at State level. • Hydro-meteorology training modules – these relate to the entry, primary and secondary validation, processing, analysis and reporting of rainfall and climate data using SWDES and HYMOS (see Table 1.1). Their contents have been largely incorporated into this Handbook as the underlying principles for data validation and analysis remain valid. 1.3 Rainfall data in groundwater studies Access to rainfall data is important in interpretation of groundwater level data, and for balancing recharge, discharge and storage of groundwater systems. GW3-DM, FM and RM describe the design, implementation, operation and maintenance of hydrometeorological networks, and rainfall data may be stored in the groundwater data processing and analysis software e-GEMS. However, subsequent data processing and analysis of rainfall data are covered only in SW8-OM and the surface water software e-SWIS has a wider range of validation and manipulation tools for rainfall data than e-GEMS. Therefore, it is recommended to carry out the majority of rainfall data processing and analysis in e-SWIS, and then export final datasets from e-SWIS, for import to e- GEMS.
  11. 11. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 5 2. The Data Management Lifecycle in HPII Agencies and staff with responsibilities for hydrometric data have a pivotal role in the development of hydro-meteorological information, through interacting with data providers, analysts and policy makers, both to maximise the utility of the datasets and to act as key feedback loops between data users and those responsible for data collection. It is important that these agencies and staff understand the key stages in the hydrometric information lifecycle (Figure 1.1), from monitoring network design and data measurement, to information dissemination and reporting. These later stages of information use also provide continuous feedback influencing the overall design and structure of the hydrometric system. While hydrometric systems may vary from country to country with respect to organisation set-ups, observation methods, data management and data dissemination policies, there are also many parallels in all stages of the cycle. 2.1 Use of hydro-meteorological information in policy and decision-making The objectives of water resource development and management in India, based on the National Water Policy and Central/State strategic plans, are: to protect human life and economic functions against flooding; to maintain ecologically-sound water systems; and to support water use functions (e.g. drinking water supply, energy production, fisheries, industrial water supply, irrigation, navigation, recreation, etc). These objectives are linked to the types of data that are needed from the HIS. SW1-DM Chapter 3.3 presents a table showing HIS data requirements for different use functions on page 19. In turn, these use functions lead to policy and decision-making uses of HIS data, such as: water policy, river basin planning, water allocation, conservation, demand management, water pricing, legislation and enforcement. Hence, freshwater management and policy decisions across almost every sector of social, economic and environmental development are driven by the analysis of hydrometric information. Its wide-ranging utility, coupled with escalating analytical capabilities and information dissemination methods, have seen a rapid growth in the demand for hydrometric data and information over the first decades of the 21st century. Central/State hydrometric agencies and international data sharing initiatives are central to providing access to coherent, high quality hydrometric information to a wide and growing community of data users. Hydrological data users may include water managers or policymakers in Central/State government offices and departments, staff and students in academic and research institutes, NGOs and private sector organisations, and hydrology professionals. An essential feature of the HIS is that its output is demand-driven, that is, its output responds to the hydrological data needs of users. SW1-FM(III) presents a questionnaire for use when carrying out a data needs assessment to gather information on the profile of data users, their current and proposed use of surface water, groundwater, hydro-meteorology and water quality data, their current data availability and requirements, and their future data requirements. Data users can, through Central/State hydrometric agencies, play a key role in improving hydrometric data, providing feedback highlighting important issues in relation to records, helping establish network requirements and adding to a centralised knowledge base regarding national data. By embracing this feedback from the end-user community, the overall information delivery of a system can be improved. A key activity within HPII was a move towards greater use of the HIS data assembled under HPI. Two examples of the use of HIS data include the Purpose-Driven Studies (PDS) and the Decision Support Systems (DSS) components of HPII. See the Hydrology Project website for more information about DSS and PDS, and access to PDS reports. The 38 PDS, which were designed, prepared and implemented by each of the Central/State
  12. 12. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 6 hydrometric agencies, are small applied research projects to investigate and address a wide range of real-world problems and cover surface water, groundwater, hydro-meteorology and water quality topics. Some examples of projects include optimisation of the river gauging station and raingauge networks in Maharashtra (PDS number SW-MH-1), and a snowmelt runoff study in the Beas basin (PDS number SW-NIH-1). The PDS utilise hydrometric data and products developed under HPI, supplemented with new data collected during HPII. Two separate DSS programmes were set up under HPII. One, for all participating implementing agencies, called DSS Planning (DSS-P), has established water resource allocation models for each State to assist them to manage their surface and groundwater resources more effectively. The other, called DSS Real-Time (DSS-RT) was specifically for the Bhakra-Beas Management Board (BBMB), although a similar DSS-RT study has also now been initiated on the Bhima River in Maharashtra. The DSS programmes have been able to utilise hydrological data assembled under the Hydrology Project to guide operational decisions for water resource management. 2.2 Hydro-meteorological monitoring network design and development Section 3.2 of this Handbook outlines the design and development of hydro-meteorological monitoring networks. Networks are planned, established, upgraded and evolved to meet a range of needs of data users and objectives, most commonly water resources assessment and hydrological hazard mitigation (e.g. flood forecasting). It is important to ensure that the hydro- meteorological, surface water, groundwater and water quality monitoring networks of different agencies are integrated as far as possible to avoid unnecessary duplication. In particular, a raingauge network should have sufficient spatial coverage that all flow monitoring stations are adequately covered. Integration of networks implies that networks are complimentary and that regular exchange of data takes place to produce high quality validated datasets. Responsibility for maintenance of Central/State hydrometric networks is frequently devolved to a regional (Divisional) or sub-regional (Sub-Divisional) level. 2.3 Data sensing and recording Sections 3.1 to 3.4 of this Handbook review hydro-meteorological monitoring networks and stations, maintenance requirements and measurement techniques. Responsibility for operation of Central/State hydro-meteorological monitoring stations is frequently devolved to a regional (Divisional) or sub-regional (Sub-Divisional) level. However, it is important that regular liaison is maintained between sub-regions and the Central/State agencies through a combination of field site visits, written guidance, collaborative projects and reporting, in order to ensure consistency in data collection and initial data processing methods across different sub-regions, maintain strong working relationships, provide feedback and influence day-to-day working practice. Hence, the Central/State agencies are constantly required to maintain a balance of knowledge between a broad-scale overview and regional/sub-region hydro-meteorological awareness. Operational procedures should be developed in line with appropriate national and international (e.g. Indian, ISO, WMO) standards (e.g. WMO Report 168 “Guide to Hydrological Practices”). For the Hydrology Project, field data from observational stations are required to be received at Sub-Divisional office level by the 5th working day of the following month (SW10-OM Protocols and Procedures). 2.4 Data validation and archival storage The quality control and long-term archiving of hydro-meteorological data represent a central function of Central/State hydrometric agencies. This should take a user-focused approach to
  13. 13. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 7 improving the information content of datasets, placing strong emphasis on maximising the final utility of data e.g. through efforts to improve completeness and fitness-for-purpose of Centrally/State archived data. Section 3.5 of this Handbook summarises the stages in the processing of hydro-meteorological data. Sections 4 to 6 of this Handbook cover the process from data entry through primary and secondary validation to correction and completion of data, and also compilation and analysis of data (Section 2.5), for rainfall, snow and climate data, respectively. During all levels of validation, staff should be able to consult station metadata records detailing the history of the site and its hydro-meteorological performance, along with topographical and isohyetal maps and previous quality control logs. Numerical and visual tools available at different phases of the data validation process, such as versatile hyetograph plotting and manipulation software to enable comparisons between different near-neighbour rainfall measurement sites, assessment of basin rainfall input hyetographs and assessment of time series statistics greatly facilitate validation. High-level appraisal by Central/State staff, examining the data in a broader spatial context, can provide significant benefits to final information products. It also enables evaluation of the performance of sub-regional data providers, individual stations or groups of stations, which can focus attention on underperforming sub-regions and encourage improvements in data quality. A standardised data assessment and improvement procedure safeguards against reduced quality, unvalidated and/or unapproved data reaching the final data archive from where they can be disseminated. However, Marsh (2002) warns of the danger of data quality appraisal systems that operate too mechanistically, concentrating on the separate indices of data quality rather than the overall information delivery function. For the Hydrology Project, the timetable for data processing is set out in SW10-OM Protocols and Procedures, and summarised in Table 2.1 of this Handbook. Data entry and primary validation of field data from observational stations is required to be completed at Sub-Divisional office level by the 10th working day of the following month (e.g. for June data by 10th working day in July), ready for secondary validation by State offices. Initial secondary validation, in State DPCs for State data, and IMD local offices for IMD data, should be completed by the end of that month (e.g. for June data by 31st July). Some secondary validation will not be possible until the end of the hydrological year when the entire year’s data can be reviewed in a long-term context, and compared with IMD data, so data should be regarded as provisional approved data until then (e.g. for June data by the end of the hydrological year plus 3 months), after which data should be formally approved and made available for dissemination to external users. At certain times of year (e.g. during the monsoon season), this data processing plan may need to be compressed, so that validated hydro- meteorological data are available sooner. 2.5 Data synthesis and analysis Central/State hydrometric agencies play a key role in the delivery of large-scale assessments of rainfall data and other climate data. Through their long-term situation monitoring, they are often well placed to conduct or inform scientific analysis at a State, National or International level, and act as a source of advice on data use and guidance on interpretation of precipitation patterns. This is especially true in the active monitoring of the State or National situation or the assessment of conditions at times of extreme events (e.g. monsoonal rains, droughts) where agencies may be asked to provide input to scientific reports and research projects, as well as informing policy decisions, media briefings, and increasing public understanding of the state of the water environment. Sections 4 to 6 of this Handbook cover compilation and analysis of data, as well as the process from data entry through primary and secondary validation to correction and completion of data (Section 2.4), for rainfall, snow and climate data, respectively.
  14. 14. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 8 Table 2.1 Hydro-meteorological data processing timetable for data for month n Activity Responsibility Deadline Rainfall, snow and climate data Data receipt Sub-Divisional office 5 th working day of month n+1 Data entry Sub-Divisional/Divisional office 10 th working day of month n+1 Primary validation Sub-Divisional/Divisional office 10 th working day of month n+1 Secondary validation State DPC State DPC Initial - end of month n+1 Final – end of hydrological year + 3 months Correction and completion State DPC State DPC Initial - end of month n+1 Final – end of hydrological year + 3 months Compilation State DPC As required Analysis State DPC As required Reporting State DPC At least annually Data requests State DPC 95% - within 5 working days 5% - within 20 working days Interagency validation IMD At least 20% of State stations, on rolling programme, by end of hydrological year + 6 months 2.6 Data dissemination and publication One of the primary functions of Central/State hydrometric agencies is to provide comprehensive access to information at a scale and resolution appropriate for a wide range of end-users. However, improved access to data should be balanced with a promotion of responsible data use by also maintaining end-user access to important contextual information. Thus, the dissemination of user guidance information, such as composite summaries that draw users’ attention to key information and record caveats (e.g. monitoring limitations, high levels of uncertainty regarding specific rainfall event accuracy, major changes in hydro-meteorological setup), is a key stewardship role for Central/State hydrometric agencies, as described in Section 7 of this Handbook. For large parts of the 20th century the primary data dissemination route for hydrometric data was via annual hardcopy publications of data tables i.e. yearbooks. However, the last decade or so has seen a shift towards more dynamic web-based data dissemination to meet the requirement for shorter lag-time between observation and data publication and ease of data re-use. Like many countries, India now uses an online web-portal as a key dissemination route for hydrometric data and associated metadata which provides users with dynamic access to a wide range of information to allow selection of stations. At least 95% of data requests from users should be processed within 5 working days. More complex data requests should be processed within 20 working days. 2.7 Real-time data During HPII many implementing agencies developed low cost real-time data acquisition systems, feeding into bespoke databases and available on agency websites. Such systems often utilise short time interval recording of data e.g. 5 minutes, 15 minutes, etc. In some instances, agencies are taking advantage of the telemetry aspect of real-time systems as a cost-effective way of acquiring data from remote locations. However, for some operational purposes (e.g. real-time flood forecasting, reservoir operation, etc), real-time data may need to be used immediately.
  15. 15. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 9 Real-time data should go through some automated, relatively simple data validation process before being input to real-time models e.g. checking that each incoming data value is within pre-set limits for the station, and that the change from preceding values is not too large. Where data fall outside of these limits, they should generally still be stored, but flagged as suspect, and a warning message displayed to the model operators. Where suspect data have been identified, a number of options are available to any real-time forecasting or decision support model being run, and the choice will depend upon the modelling requirements. Whilst suspect data could be accepted and the model run as normal, it is more common to treat suspect data as missing or to substitute them with some form of back-up, interpolated or extrapolated data. This is necessary for hydrometric agencies to undertake some of their day-to-day functions and, in such circumstances, all the data should be thoroughly validated as soon as possible, according to the same processing timetable and protocols as other climate data. Real-time data should also be regularly transferred to the e-SWIS database system, through appropriate interfaces, in order to ensure that all hydro-meteorological data are stored in a single location and provide additional back-up for the real-time data, but also to provide access to the data validation tools available through the eSWDES and eHYMOS modules of e-SWIS.
  16. 16. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 10 3. Hydro-Meteorological Monitoring Stations and Data 3.1 Types of hydro-meteorological monitoring station SW3-FM(I) Chapter 2.1 lists different types of hydro-meteorological stations and instruments which measure various rainfall and climate variables. For each of these, Table 3.1 (two parts) lists the relevant section in the HIS Manual SW for detailed information on design and installation, maintenance, measurement, data entry, primary and secondary validation, correction and completion of data, compilation and analysis of data, and reporting. Stations include: • SRG – a rainfall station equipped with a standard or non-recording raingauge. Additional information on design and installation (Table 3.1) includes the appropriate capacity of raingauge containers for different Indian States. • ARG – a rainfall station with an autographic or recording raingauge, which will also have an SRG for check purposes. • TBR – a tipping bucket raingauge is a type of ARG, often connected to a data logger, which will also have an SRG for check purposes. Additional information on design and installation (Table 3.1) includes the relative advantages of ARGs and TBRs, compared to each other and to SRGs. • Snow stations – a type of station not included in HPI, where observation are made of:  Snowfall since the last observation;  Total depth of snow on the ground (i.e. the depth of the snowpack);  Snow-water equivalent (SWE i.e. the depth of liquid precipitation contained in that snowfall and/or the snowpack). • FCS – a full climate station where, in addition to rainfall, a comprehensive range of other climate variables are observed for direct measurement of evaporation and/or for indirect estimation of evaporation:  Pan-evaporation (direct measurement) using a pan-evaporimeter;  Temperature of pan water;  Sunshine duration using a sunshine recorder;  Air temperature using thermometers and optional thermograph;  Humidity using thermometers and optional hygrograph;  Wind speed and direction using an anemometer and wind vane;  Atmospheric pressure using a barometer and/or barograph. • AWS – an automatic weather station is an FCS where the climate variables are observed by automatic/recording means. These were not included in HPI documentation. AWS were not included in HPI. A set of specifications for hydrometric equipment was compiled under HPI and updated under HPII. The specifications, which are downloadable from the Hydrology Project website, provide a guideline for procurement (with examples of some procurement templates and documents also on the Hydrology Project website). 3.2 Hydro-meteorological monitoring networks Monitoring networks should be considered to be dynamic entities. It is important that the current utility of well-established monitoring networks is periodically assessed to ensure that they continue
  17. 17. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 11 Table 3.1 Where to go in the HIS Manual SW for hydro-meteorological data management guidance: rainfall and snow Instrument / Variable Design & Installation Maintenance Measurement Data entry Primary Validation Secondary Validation Correction & Completion Compilation Analysis Reporting SRG SW3-DM 6.2.1, 8.2.2 SW3-FM(II) 1.3 SW3-FM(V) 2.2, 3.2 SW3-FM(II) 1.2 SW8-OM(I) 4.4, 4.5 SW8-OM(I) 5 SW8-OM(II) 2 SW8-OM(II) 3 SW8-OM(II) 4 SW8-OM(III) 4 SW8-OM(III) 9 ARG SW3-DM 6.2.2, 8.2.3 SW3-FM(III) 2.3 SW3-FM(V) 2.2, 3.3 SW3-FM(III) 2.2.2 SW8-OM(I) 4.6 SW8-OM(I) 5 SW8-OM(II) 2 SW8-OM(II) 3 SW8-OM(II) 4 SW8-OM(III) 4 SW8-OM(III) 9 TBR SW3-DM 6.2.3, 8.2.4 SW3-FM(III) 3.3 SW3-FM(V) 2.2, 3.4 SW3-FM(III) 2.3.2 SW8-OM(I) 4.7 SW8-OM(I) 5 SW8-OM(II) 2 SW8-OM(II) 3 SW8-OM(II) 4 SW8-OM(III) 4 SW8-OM(III) 9 Snow Not included in HPI
  18. 18. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 12 Table 3.1 Where to go in the HIS Manual SW for hydro-meteorological data management guidance: climate and evaporation Instrument / Variable Design & Installation Maintenance Measurement Data entry Primary Validation Secondary Validation Correction & Completion Compilation Analysis Reporting FCS SW3-DM 6.3, 8.3 SW8-OM(I) 6.4, 6.5 SW8-OM(I) 7 SW8-OM(II) 5 SW8-OM(II) 6 SW8-OM(II) 5 SW8-OM(III) 11 Pan- evaporation SW3-DM 6.3.5, 8.3.7 SW3-FM(IV) 5.2 SW3-FM(V) 2.2, 3.9 SW3-FM(IV) 5.2 SW8-OM(I) 7.8.2 SW8-OM(I) 6.4, 6.5 SW8-OM(I) 7.8 SW8-OM(II) 5 SW8-OM(II) 6.7 SW8-OM(II) 5.2 SW8-OM(III) 11 Penman evapotran- spiration SW3-DM 2.3 SW8-OM(II) 5.3 SW8-OM(III) 11 Sunshine duration SW3-DM 6.3.1, 8.3.8 SW3-FM(IV) 6.3 SW3-FM(V) 2.2, 3.10 SW3-FM(IV) 6.2 SW8-OM(I) 7.7.2 SW8-OM(I) 6.7 SW8-OM(I) 7.7 SW8-OM(II) 5 SW8-OM(II) 6.6 Temperature SW3-DM 6.3.2, 8.3.4-5 SW3-FM(IV) 3.1.3, 3.2.3 SW3-FM(V) 2.2, 3.6, 3.7 SW3-FM(IV) 3.1.2, 3.2.2 SW8-OM(I) 7.3.2 SW8-OM(I) 6.4, 6.5, 6.6 SW8-OM(I) 7.3 SW8-OM(II) 5 SW8-OM(II) 6.2 Humidity SW3-DM 6.3.3, 8.3.6 SW3-FM(IV) 4.2.3 SW3-FM(V) 2.2, 3.8 SW3-FM(IV) 4.1, 4.2.2 SW8-OM(I) 7.3.2, 7.4.2 SW8-OM(I) 6.4, 6.5, 6.6 SW8-OM(I) 7.3, 74 SW8-OM(II) 5 SW8-OM(II) 6.3 Wind speed & direction SW3-DM 6.3.4, 8.3.3 SW3-FM(IV) 1.1.3, 1.2.3 SW3-FM(V) 2.2, 3.5 SW3-FM(IV) 1.1.2, 1.2.2 SW8-OM(I) 7.5.2 SW8-OM(I) 6.4, 6.5 SW8-OM(I) 7.5 SW8-OM(II) 5 SW8-OM(II) 6.4 Atmospheric pressure SW3-DM 6.3.6, 8.3.9 SW3-FM(IV) 7.2 SW3-FM(IV) 7.2 SW8-OM(I) 7.6.2 SW8-OM(I) 6.6 SW8-OM(I) 7.6 SW8-OM(II) 5 SW8-OM(II) 6.5 AWS Not included in HPI
  19. 19. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 13 to meet changing requirements and to optimise the information they deliver. Network reviews should be done in collaboration with other agencies. SW3-FM(I) Chapter 1 and SW3-DM Chapter 3 describe network design and optimisation for monitoring rainfall and other climate variables. This is a multi-step process comprising: 1. Identification of hydrological data users and their data needs to understand what data are required and at what frequency. 2. Definition of the purposes and objectives of the network in order to fulfill the hydrological data need, and evaluation of the consequences of not meeting those targets, to inform a prioritisation of objectives in case of budget constraints. 3. Evaluation of the existing and required network densities, using an effectiveness measure which takes into account the spatial and temporal correlation of the variables. This step and steps 4 and 5 may involve the development of regionalisation and network optimisation techniques (e.g. Institute of Hydrology, 1999; Hannaford et al., 2013). 4. Evaluation of the existing network versus the required one in relation to network density, purposes and objectives, distribution with respect to surface water and groundwater monitoring networks, adequacy of existing equipment and operational procedures, and possible improvements to existing network. 5. Site and equipment selection i.e. the identification of gaps in the existing network if it is inadequate to meet the purposes and objectives. This may require the collection of maps and background information to inform the revised network design. 6. Estimation of overall costs of installing, operating and maintaining the existing and new sites. Achieving an optimum network design may involve an iterative process, repeating some or all of steps 3 to 6, until a satisfactory outcome is reached. 7. Preparation of phased implementation plan for optimum network that is prioritised, realistic and achievable in the time scales allowed. 8. Selection of sites. SW3-FM(I) devotes Chapter 2 to this topic, identifying the factors that should be taken into consideration to ensure long-term reliable data. These include: technical (positioning to minimise estimation errors and optimise integration with surface water and groundwater networks); environmental (topography around site, exposure conditions at site, future development near site, vegetation at and near site, proximity of water bodies, no water- logging at site); logistical (accessibility, communication, staffing); security (location of site, design of site, staffing); legal (land acquisition, rights of passage); and financial (cost of land, cost of civil works, equipment procurement, calibration and maintenance, operating costs, staffing) aspects. Site selection should be carried out in collaboration with IMD and should involve a site visit, which may reveal that the desired location is unsuitable, and an alternative site may need to be considered. The selection of appropriate locations for snow stations (also known as snow courses) may be challenging because of terrain and wind effects. In flat, open areas, it is desirable to have snow stations in typical landscapes, such as in open fields and forests, with different snow accumulation conditions. In mountainous areas, additional criteria may apply: • At sites sufficiently accessible to ensure continuity of surveys • At elevations and exposures where there is little or no melting prior to the peak accumulation • At a site having protection from strong wind movement
  20. 20. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 14 • In forested areas where the sites can be located in open spaces sufficiently large so that snow can fall to the ground without being intercepted by the trees 9. Establishment of a framework for periodic network reviews (e.g. after 3 years or sooner if new data needs develop) i.e. starting this process again from step 1. As an example of the theory and practical aspects of network design, SW3-RM presents a pilot study for designing a raingauge network for two sub-basins of the Mahanadi river basin in Orissa. A good example of a monitoring network review under HPII is the Purpose Driven Study (PDS) on optimisation of the river gauging station and raingauge networks in Maharashtra (PDS number SW-MH-1). For more detailed information see: SW2-DM Chapter 7 which provides some generic guidance on types of network and the steps in network design; SW2-DM Chapters 3.2.1 to 3.2.6 which describe classification of stations and offer some examples of types of network; and Hydro-Meteorology Training Module 45 “How to review monitoring networks”. 3.3 Site inspections, audits and maintenance Regular maintenance of equipment, together with periodic inspections and audits, ensures collection of good quality data and provides information that may assist in future data validation queries. Table 3.1 lists the relevant section in the HIS Manual SW for maintenance of the different types of hydro-meteorological stations and instruments. Whilst this topic is largely covered in different parts of SW3-FM(II)-(V), information is collated together in the document “Surface Water O&M norms” which is a maintenance guide for hydro-meteorology, stage-discharge and water quality instrumentation and equipment. Maintenance and calibration requirements depend to a large extent of the type of station, instruments and equipment so are often site-specific. A supply of appropriate spare parts should be kept on site and/or taken on station visits in case they are needed. SW3-FM(V) Annex II lists maintenance norms for hydro-meteorological stations, including maintenance of civil works, maintenance of equipment, costs of consumable items and payments to staff (where the costs should be regarded as out of date). Formal inspections cum audits are carried out, and station log sheets completed, at a frequency dependent on the importance of the station, the type of station and the time of year and will typically vary between monthly and annually as set out in SW3-FM(V) Chapter 1, with station log sheets for inspections of rainfall and climate stations in FM(V) Annex I. Activities may include: checking the performance of and motivating the field staff; and identifying existing or potential problems with the site, instruments, equipment and observation procedures at an early stage so they can be rectified. However, a brief inspection of the site and instrumentation should be made every day that somebody is on site. 3.4 Data sensing and recording Table 3.1 lists the relevant section in the HIS Manual SW for operational instructions on the measurement of rainfall and other climate variables at hydro-meteorological stations. Note that there is some overlap between SW3-FM and SW3-DM, and between the network design and site selection topic (covered in Section 3.2 of this Handbook) and data measurement. See also the document “Illustrations: hydrological observations” which demonstrates how to make measurements of rainfall, water level and flow at stations, and also how to carry out an inspection at those stations.
  21. 21. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 15 At an FCS, the observations of the various climate variables are made once or twice a day in a prescribed sequence commencing from 10 minutes preceding the scheduled hours i.e. 08:20 for 08:30 readings, and 17:20 for 17:30 readings. The sequence is: wind instruments, raingauges, thermometers, evaporation, radiation, and culminating with atmospheric pressure at exactly 08:30 and 17:30. Charts on autographic instruments are changed daily during the 08:30 reading, except the sunshine recorder card which is changed during the 17:30 reading or after sunset, whichever is later. Hourly values are abstracted from autographic charts and tabulated daily. Observers at an FCS should also make observations of the depth of snow on the ground, should it occur. For stations with only an SRG, the measurement is made at 08:30 IST only, though more frequent observations are required during heavy rain to avoid overflow due to the limited capacity of the raingauge container. SRGs at FCSs are read twice daily along with the other climate variables. ARG chart recorders are digitised at 1-hour or 15-minute time intervals depending on what is most appropriate for the location and the intensity of the rainfall (SW3-DM Chapter 5.2). TBRs with data loggers can operate in time mode where the number of tips in a pre-set time interval (e.g. 1 hour, 15 minutes, etc) are recorded, or in event mode where the times of every tip are recorded, thereby producing a more flexible record for subsequent analysis. However, note that event mode data cannot currently be stored in e-SWIS. At snow stations, observations are made of the snowfall since the last observation, the total depth of snow on the ground (i.e. the depth of the snowpack) and the snow-water equivalent (SWE i.e. the depth of liquid precipitation contained in that snowfall and/or the snowpack). Measurements may be made daily or sub-daily. The accuracy of measurements of snowfall, snow depth and SWE depends on the graduations of the scales being used, and on instrumental and subjective errors. At some snow stations, data are augmented by regular measurements of sunshine, temperature, humidity, wind speed and direction, and atmospheric pressure. The extent of snow cover is usually made from one or a combination of field observations, aerial survey data and satellite imagery. The observer should always note any occurrences which may influence the climate variables as observed by the instruments. These may include: damage to the equipment for a specified reason. The observer should also note any maintenance activities carried out at the monitoring site (e.g. change batteries, clean sensor, etc). The observer should double-check that that any manual reading is taken correctly, and transcribed correctly (e.g. decimal point in right place). If the reading is later transferred to another document (e.g. hand copied or typed in, or abstracted from a chart), the observer should always check that this has been done correctly. An experienced and suitably qualified observer should compare measurements with equivalent ones from earlier that day or from the day before, if available, as an additional form of checking. However, the observer should not, under any circumstances, retrospectively alter earlier readings or adjust current readings, but should simply add an appropriate comment. Data collected in the field are delivered to a Data Processing Centre (DPC) on a variety of media, including handwritten forms and notebooks, charts and digital data. 3.5 Data processing SW8-OM(IV) Chapter 2 sets out the steps in processing of hydro-meteorological data, which starts with preliminary checking in the field, as described in Section 3.4 of this Handbook, through receipt of raw field data at a DPC, through successively higher levels of validation in State and Central DPCs, before data are fully validated and approved in the National database. Validation ensures that the data stored are as complete and of the highest quality as possible by: identifying errors and sources of errors to mitigate them occurring again, correcting errors where possible, and
  22. 22. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 16 assessing the reliability of data. It is important for staff to be aware of the different errors that may occur as described in SW8-OM(IV) Chapter 2.5.1. Hydro-meteorological data validation is split into two stages: primary and secondary. Validation is very much a two-way process, where each step feeds back to the previous step any comments or queries relating to the data provided. The diverse hydrological environments found in India mean that staff conducting data validation should be familiar with the expected climate in order to identify potentially anomalous behaviour. SW3-RM presents some tabular and graphical summary data for selected coastal and hill FCSs in the HPI Indian States for the period 1931-60, to give an insight into typical annual variation of the climate variables. The data processing steps comprise: 1. Receipt of data according to prescribed target dates. Rapid and reliable transfer of data is essential, using the optimal method based on factors such as volume, frequency, speed of transfer/transmission and cost. Maintenance of a strict time schedule is important because it gives timely feedback to monitoring sites, it encourages regular exchanges between field staff, Sub-Divisional offices, State and Central agencies, it creates continuity of processing activities at different offices, and it ensures timely availability of final (approved) data for use in policy and decision-making. 2. Entry of data to computer, using the eSWDES module of e-SWIS, is primarily done at a Sub- Divisional office level where staff are in close contact to field staff who have made the observations and/or collected the chart or digital data. Historical data, previously only available in hardcopy form, may also be entered this way. Each Central/State agency should have a programme of historical data entry. 3. Primary data validation which should be carried out in State DPCs for State data and IMD local offices for IMD data, as soon as possible after the observations are made, data extracted from charts, or data downloaded from loggers, using the eSWDES module of e-SWIS. This ensures that any obvious problems (e.g. indicating an instrument malfunction, observer error, etc) are spotted at the earliest opportunity and resolved. Other problems may not become apparent until more data have been collected, and data can be viewed in a longer temporal context during secondary validation. 4. Secondary data validation which should be carried out in State DPCs for State data and IMD local offices for IMD data, to take advantage of the information available from a large area by focusing on comparisons with the same variable at other good quality, nearby monitoring sites (analogue stations) which are expected to exhibit similar hydrological behaviours (e.g. comparison of cumulative rainfall from two raingauges), uses the eHYMOS module of e-SWIS. States should have access to IMD data during secondary validation, and may receive support from IMD in this activity. 5. Data correction and completion are elements of data validation which are used to infill missing value, sequences of missing values or correct clearly erroneous values, in order to make the time series as complete as possible. Some suspect (doubtful) data values may still justifiably remain after this stage if correction is not possible so that the original data value remains the best estimate of the true value for that day and time. 6. Data storage. The e-SWIS HIS database, of both approved data and unapproved data undergoing primary and secondary validation, is backed up automatically. Therefore, there is no need to make regular back-ups, unless any data are stored outside the HIS database, for instance in Excel files or other formats awaiting data entry, or in stand-alone real-time databases – such files should be securely backed up, ideally onto an external back-up device and/or backed up network server, so that there is no risk of data loss. All PCs should have up- to-date anti-virus software.
  23. 23. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 17 Raw field data, in the form of handwritten forms and notebooks, and charts should also be stored in a secure manner after database entry to ensure that original field data remain available should any problems be identified during validation and analysis. Such hardcopy data should ultimately be securely archived, in the State DPC for State data or IMD local office for IMD data, possibly by scanning documents and storing them digitally. 7. Interagency data validation by IMD – IMD should aim to validate at least 20% of current and historic data from State hydro-meteorological monitoring stations every year, on a rolling programme, so that IMD has independently validated the data from every State gauge at least once every 5 years. Interagency validation is a 2-way process and IMD should discuss any identified issues and agree final datasets with State DPCs through a 2-way consultative process, to build capacity for data validation within the States. For rainfall, snow and climate data, Sections 4 to 6 of this Handbook, respectively, cover the process from data entry through primary and secondary validation to correction and completion of data, and also compilation (i.e. the transformation of data observed at one time interval to another time interval e.g. aggregation from daily rainfall to monthly rainfall) and analysis of data.
  24. 24. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 18 4. Rainfall Data Processing and Analysis 4.1 Data entry 4.1.1 Overview Entry of data to computer is primarily done at a Sub-Divisional office level where staff are in close contact to field staff who have made the observations and/or collected the chart or digital data. Data entry is carried out using e-SWIS, the data entry module of which replicates the SWDES software from HPI, and is referred to as eSWDES. Prior to entry to computer, two manual activities are essential: registration of receipt of the data, and manual inspection of the rainfall charts, forms and notebooks from the field, for complete information and obvious errors. Data entry (see Table 3.1) and primary validation of field data from observational stations is required to be completed at Sub-Divisional office level by the 10th working day of the following month (e.g. for June data by 10th working day in July), ready for secondary validation by State offices. 4.1.2 Manual inspection of field records Prior to data entry to computer an initial inspection of field records is required. This is done in conjunction with notes received from the observation station on equipment problems and faults, missing records or exceptional rainfall. Rainfall sheets and charts are inspected for the following: • Is the station name and code and month and year recorded? • Does the number of record days correspond with the number of days in the month? • Are there some missing values or periods for which rainfall has been accumulated during absence of the observer? • Have monthly totals of rainfall and rain days been entered? • Have the autographic rainfall hourly totals been extracted? • Is the record written clearly and with no ambiguity in digits or decimal points? Any queries arising from such inspection should be communicated to the observer to confirm ambiguous data before data entry. Any unresolved problems should be noted and the information sent forward with the digital data to Divisional/State offices to assist in initial secondary validation. Any equipment failure or observer problem should be communicated to the supervising field officer for rectification. 4.1.3 Entry of daily rainfall data Using the eSWDES module in e-SWIS, the user selects the correct station and daily series. The screen for entry (or editing) of daily rainfall is displayed, along with the upper warning level used to flag suspect values (which can be altered for different seasons), and the maximum and minimum values for that station. For rainfall the minimum value is 0.0 mm, and a rainy day is defined as that day on which the rainfall is more than 0.0 mm. The user selects the correct year and month, and enters the daily rainfall value recorded at 08:30 for each date, adding comments where appropriate. Negative and non-numerical entries are automatically rejected. For each month, the user also enters the number of rain days, total rainfall and maximum rainfall. The software also calculates the number of rain days, the cumulative rainfall and the maximum rainfall as the user enters the data. Two types of data entry checks are performed for this case of daily rainfall data: • The number of rain days, total monthly rainfall and maximum rainfall entered by the user are compared with the values calculated by the software. In the case of a mismatch the user is
  25. 25. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 19 prompted by colour highlighting and can refer back to the field documents to see if there was some error in entering the data. If cumulated values are also available in the field documents, it becomes quicker to isolate the error. • The entered daily data are compared against the upper warning level and the maximum limit. This identifies potentially suspect values to the user who can refer back to the field documents to see if there was some error in entering the data. If values which exceed the upper warning level and the maximum limit are actually reported in the field documents, the user should add an appropriate comment. Any mismatch remaining after thorough checking of the field documents must be due to incorrect field computations by the observer and should be communicated to the supervising field officer. The user should also view entered data graphically to identify potentially suspect data not apparent in tabular form, which may reflect an error in data entry. There are three ways in which the entered data can be plotted: daily data for the month, daily data for the year, and monthly totals for the year. Missing data When data are missing, the corresponding cell is left as -999 (not zero) and a comment entered against that day. Accumulated data Where the observer has missed readings over a period of days and an accumulated total is subsequently measured, the cells corresponding to the missed days are left as -999 (not zero) and a comment entered against the date of the accumulation to specify the period over which the accumulation has occurred (e.g. Accumulated from 23 to 27 Sep). There are occasions when the observer is legitimately absent from her/his station, for example on account of sickness. The observer should be encouraged to leave such spaces “Missing” or “Accumulated” rather than guess the missing values. The completion procedures (Section 4.4), based on adjoining information, are better able to estimate such missing values. 4.1.4 Entry of rainfall data at twice daily interval Using the eSWDES module in e-SWIS, the user selects the correct station and twice-daily series. The screen for entry (or editing) of twice-daily rainfall is displayed, along with the upper warning level used to flag suspect values (which can be altered for different seasons), and the maximum and minimum values for that station. For rainfall the minimum value is 0.0 mm, and a rainy day is defined as that day on which the rainfall is more than 0.0 mm. The user selects the correct year and month, and enters the twice-daily rainfall values recorded at 17:30 the previous day and 08:30 for each date, adding comments where appropriate. Negative and non-numerical entries are automatically rejected. For each month, the user also enters the number of rain days, total rainfall and maximum rainfall. The software also calculates the number of rain days, the total daily rainfall, the cumulative rainfall and the maximum rainfall as the user enters the data. Two types of data entry checks are performed for this case of twice-daily rainfall data: • The number of rain days, total daily rainfall, total monthly rainfall and maximum rainfall entered by the user are compared with the values calculated by the software. In the case of a mismatch the user is prompted by colour highlighting and can refer back to the field documents to see if there was some error in entering the data. If cumulated values are also available in the field documents, it becomes quicker to isolate the error. • The entered daily data are compared against the upper warning level and the maximum limit. This identifies potentially suspect values to the user who can refer back to the field documents to see if there was some error in entering the data. If values which exceed the upper warning level and the maximum limit are actually reported in the field documents, the user should add an appropriate comment.
  26. 26. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 20 Any mismatch remaining after thorough checking of the field documents must be due to incorrect field computations by the observer and should be communicated to the supervising field officer. The user should also view entered data graphically to identify potentially suspect data not apparent in tabular form, which may reflect an error in data entry. There are four ways in which the entered data can be plotted: twice daily data for the month, daily data for the month, daily data for the year, and monthly totals for the year. Missing and accumulated data are handled in the same way as for entry of daily rainfall data (Section 4.1.3). 4.1.5 Entry of hourly rainfall data Hourly rainfall data are obtained either from the chart records of ARGs or from the digital data of TBRs. Digital data can also be imported directly, but can undergo entry checks and be viewed graphically using this option. Using the eSWDES module in e-SWIS, the user selects the correct station and hourly series. The screen for entry (or editing) of hourly rainfall is displayed, along with the upper warning level used to flag suspect values (which can be altered for different seasons), and the maximum and minimum values for that station. For rainfall the minimum value is 0.0 mm. The user selects the correct year and month, and enters the hourly rainfall values, with each row corresponding to a different day and each column to a different time, adding comments where appropriate. The rainfall value is entered against the time following the hour in which the rainfall occurred e.g. rainfall falling and recorded from 11:30 to 12:30 is recorded against 12:30. Negative and non-numerical entries are automatically rejected. For each day, the user enters the daily total. For each month, the user also enters the columnar total for each hourly period, the number of rain days, total rainfall and maximum rainfall. The software also calculates the daily and hourly totals, the number of rain days, the cumulative rainfall and the maximum rainfall as the user enters the data. Two types of data entry checks are performed for this case of hourly rainfall data: • The number of rain days, columnar total for each hourly period, total daily rainfall, total monthly rainfall and maximum rainfall entered by the user are compared with the values calculated by the software. In the case of a mismatch the user is prompted by colour highlighting and can refer back to the field documents to see if there was some error in entering the data. If cumulated values are also available in the field documents, it becomes quicker to isolate the error. • The entered hourly data are compared against the upper warning level and the maximum limit. This identifies potentially suspect values to the user who can refer back to the field documents to see if there was some error in entering the data. If values which exceed the upper warning level and the maximum limit are actually reported in the field documents, the user should add an appropriate comment. Any mismatch remaining after thorough checking of the field documents must be due to incorrect field computations by the observer and should be communicated to the supervising field officer. The user should also view entered hourly data for the day and month graphically to identify potentially suspect data not apparent in tabular form, which may reflect an error in data entry. Missing and accumulated data are handled in the same way as for entry of daily rainfall data (Section 4.1.3).
  27. 27. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 21 4.1.6 Import/entry of digital data Digital data from TBRs can take two forms: time mode where the number of tips in a pre-set time interval (e.g. 1 hour, 15 minutes, etc) are recorded, or in event mode where the times of every tip are recorded, thereby producing a more flexible record for subsequent analysis. TBR data can be imported directly should an appropriate import interface be available (bespoke to each type of data logger), and can undergo entry checks and be viewed graphically as described in Section 4.1.5. Time mode data are imported to an appropriate equidistant time series, whilst event mode data are imported to a non-equidistant time series. 4.2 Primary validation 4.2.1 Overview Primary validation is primarily done at a Sub-Divisional office level where staff are in close contact to field staff who have made the observations and/or collected the chart or digital data. Primary validation is carried out using e-SWIS, the data entry module of which replicates the SWDES software from HPI, and is referred to as eSWDES. Primary validation (see Table 3.1) of field data from observational stations is required to be completed at Sub-Divisional office level by the 10th working day of the following month (e.g. for June data by 10th working day in July), ready for secondary validation by State offices. This time schedule ensures that any obvious problems (e.g. indicating an instrument malfunction, observer error, etc) are spotted at the earliest opportunity and resolved. Other problems may not become apparent until more data have been collected, and data can be viewed in a longer-term context during secondary validation. Primary validation of rainfall data focuses on validation within a single data series by making comparisons between individual observations and pre-set physical limits, and between two measurements of rainfall at a single station (e.g. daily rainfall from an SRG and accumulated daily rainfall from an ARG/TBR). The high spatial and temporal variability of rainfall data compared to other climate variables makes validation of rainfall more difficult. This is particularly the case on the Indian sub-continent, experiencing a monsoon-type climate involving convective precipitation. Examples of many of the techniques described in this section are given in Hydro-Meteorology Training Module 08 “How to carry out primary validation for rainfall data” and Training Module 10 “How to correct and complete rainfall data”. 4.2.2 Typical errors Staff should be aware of typical errors in rainfall measurement, listed in Table 4.1, and these should be considered when interpreting data and possible discrepancies (SW8-OM(I) Chapter 5.2). SRG errors from most of these sources are very difficult to detect from the single record of the standard raingauge, unless there has been a gross error in reading or transcribing the value (Section 4.2.3). Errors are more readily detected if there is a concurrent record from an ARG or TBR. As these too are subject to errors (of a different type), comparisons with the SRG are very important (Section 4.2.4). The final check by comparison with raingauges at neighbouring stations should show up further anomalies, especially for those stations which do not have an ARG or TBR at the site. This is carried out during secondary validation where more gauges are available for comparison (Section 4.3). 4.2.3 Comparison with upper warning level and maximum and minimum limits Both hourly and daily rainfall data should be validated against physical limits, which are required to be quite wide to avoid the possibility of rejecting true extreme values. For rainfall data, the minimum limit is 0.0 mm. The maximum limit will vary spatially over India with climatic region and
  28. 28. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 22 Table 4.1 Measurement errors for rainfall data SRG measurement errors • Observer reads measuring glass incorrectly • Observer enters amount incorrectly in the field sheet • Observer reads gauge at the wrong time (i.e. the correct amount may thus be allocated to the wrong day) • Observer enters amount to the wrong day • Observer uses wrong measuring glass (i.e., 200 cm 2 glass for 100 cm 2 gauge, giving half the true rainfall, or 100 cm 2 glass for 200 cm 2 gauge giving twice the true rainfall • Observed total exceeds the capacity of the gauge • Instrument fault - gauge rim damaged so that collection area is affected • Instrument fault - blockage in raingauge funnel so that water does not reach collection bottle and may overflow or be affected by evaporation • Instrument fault - damaged or broken collector bottle and leakage from gauge ARG measurement errors • Potential measurement faults are primarily instrumental rather than caused by the observer • Funnel is blocked or partly blocked so that water enters the float chamber at a different rate from the rate of rainfall • Float is imperfectly adjusted so that it syphons at a rainfall volume different from 10 mm • In very heavy rainfall the float rises and syphons so frequently that individual pen traces cannot be distinguished • Clock stops; rainfall not recorded or clock is either slow or fast and thus timings are incorrect • Float sticks in float chamber; rainfall not recorded or recorded incorrectly • Observer extracts information incorrectly from the pen trace TBR measurement errors • Funnel is blocked or partly blocked so that water enters the tipping buckets at a different rate from the rate of rainfall • Buckets are damaged or out of balance so that they do not record their specified tip volume • Reed switch fails to register tips • Reed switch double registers rainfall tips as bucket bounces after tip. (better equipment includes a debounce filter to eliminate double registration) • Failure of electronics due to lightning strike etc. (though lightning protection usually provided) • Incorrect set up of measurement parameters by the observer or field supervisor orography, and maximum limits for 1-hour and 1-day rainfall should be based of analysis of historic data for the station or IMD maps of 1-hour maximum rainfall and 1-day maximum rainfall. However, validation of rainfall data against a maximum value does not discriminate those comparatively frequently occurring erroneous data which are less than the prescribed maximum limit. In view of this, it is advantageous to consider an upper warning level, which can be employed to filter high data values which are not expected to occur frequently. For daily rainfall data, this limit can be set statistically e.g. to 99th percentile of actual rainfall values excluding zero values. A similar statistic can be employed for obtaining a suitable upper warning level for hourly rainfall data. Setting such warning levels and limits for each station ensures rapid filtering of potentially erroneous values, and should result is not too many and not too few, data values being flagged as suspect. During data entry, hourly and daily data are compared against the upper warning level and the maximum limit. If values which exceed the upper warning level and the maximum limit are actually reported in the field documents, the user should have considered the values suspect and added an appropriate comment, for further attention during primary validation. Data entries which are more than the prescribed upper warning level or the maximum limit may imply that the earlier maximum has indeed been crossed. In such cases, it is expected that there will be a few nearby stations recording similar higher rainfall to support such inferences, and this should be reviewed during secondary validation.
  29. 29. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 23 4.2.4 Comparison of daily and sub-daily rainfall data For stations with an ARG or a TBR, an SRG is always also available. Thus, daily rainfall data are available from two independent sources. The accumulated daily rainfall from the sub-daily raingauge should be compared with the daily rainfall at the daily raingauge, to check for consistency. Differences which are less than 5% can be attributed to exposure, instrument accuracy and precision in tabulating the analogue records and are ignored, but differences greater than 5% may indicate potential errors and should be examined further. The observation made using the SRG is traditionally regarded as comparatively more reliable. This is based on the assumption that there is higher degree of possibility of malfunctioning of autographic or digital recorders owing to their mechanical and electronic systems. However, significant systematic or random errors are also possible in the SRG as shown in Table 4.1. If the error is in the autographic or digital records, then it should be possible to relate it either to instrumental or observational errors. Moreover, such errors tend to repeat under similar circumstances. Comparison of daily and accumulated sub-daily rainfall data may be carried out in tabular or graphical form, with an additional table column for those days where the discrepancy is more than 5%, or a second graph axis showing percent discrepancy: • Where the recording gauge gives a consistently higher or lower total than the daily gauge, then the recording gauge could be out of calibration and either tipping buckets (TBR) or floats (ARG) need recalibration → Accept SRG and adjust ARG or TBR • Where agreement is generally good but difference increases in high intensity rainfall suggests that for the ARG either (i) the syphon is working imperfectly in high rainfall or (ii) the chart trace is too close to distinguish each 10 mm trace (underestimate by multiples of 10 mm), and for the TBR (i) the gauge is affected by bounce sometimes giving double tips → Accept SRG and adjust ARG or TBRG • Where a day of positive discrepancy is followed by a negative discrepancy and rainfall at the recording gauge was occurring at the observation hour, then it is probable that the observer read the SRG at a different time from the ARG. The sum of SRG readings for successive days should equal the 2-day total for the ARG or TBR → Accept ARG or TBR and adjust SRG • Where the agreement is generally good but isolated days have significant differences, then the entered hourly data should be checked against the field documents. Entries resulting from incorrect entry are corrected. Check that water added to a TBR for calibration is not included in rainfall total. Otherwise there is probable error in the SRG observation → Accept ARG or TBR and adjust SRG • Where the values reported for the daily rainfall by the SRG and ARG/TBR correspond exactly for considerable periods, it is conceivable that the observer has forcefully manipulated one or both datasets. Variation must exist due to variance in the catch and instrument and observation variations. Both hourly and daily data should be checked against the field documents to attempt to ascertain which has been manipulated. In the case where the SRG record is accepted records from autographic gauges at neighbouring stations can be used in conjunction with the SRG at the station to correct the ARG/TBR record at the station. This involves hourly distribution of the daily total from the SRG at the station by reference to the hourly distribution at one or more neighbouring stations. Donor (or base) stations are selected by making comparison of cumulative plots of events in which autographic records are available at both stations and selecting the best available for estimation. If the daily rainfall on the day in question at the station under consideration is Dtest and the hourly rainfall for the same daily period at the selected neighbouring donor station are Hbase,i (i = 1, 24), the hourly rainfall at the station under consideration, Htest,i are obtained as:
  30. 30. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 24 The procedure may be repeated for more than one neighbouring donor station and the average or resulting hourly totals calculated. In the case where the ARG/TBR record is accepted it can be used to correct the SRG record at the station. The SRG data values are made equal to the accumulated daily or twice-daily totals from the ARG/TBR. Where a shift in one of the data series has been detected, this can be removed after looking at the field documents and corresponding tabulated data and adjusting the data appropriately. The adjusted data should be flagged as corrected and an appropriate comment added. Where a doubtful or incorrect rainfall value is identified, and there is any uncertainty as to the correct action, this should be marked with an appropriate flag to indicate that it is suspect. The data flagged as suspect are reviewed at the time of secondary validation. 4.3 Secondary validation 4.3.1 Overview Secondary validation of rainfall data is primarily carried out at State DPCs, to take advantage of the information available from a larger area. Secondary validation is carried out using e-SWIS, the validation module of which replicates the HYMOS software from HPI, and is referred to as eHYMOS. Data may also be exported to Excel for secondary validation. For the Hydrology Project, secondary validation (see Table 3.1) done at State level should be completed by the end of the following month (e.g. for June data by 31st July). Some secondary validation (including comparison with IMD data) will not be possible until the end of the hydrological year when the entire year’s data can be reviewed in a long-term context, so data should be regarded as provisional approved data until then (e.g. for June data by the end of the hydrological year plus 3 months), after which data should be formally approved and made available for dissemination to external users. Data entering secondary validation have already received primary validation on the basis of knowledge of the station and instrumentation and field documents. Data may have been flagged as missing, accumulated, shifted or suspect for some other reason e.g. a mismatch in the number of rain days. Secondary validation focuses on comparisons with neighbouring stations to identify suspect values. However, data processing staff should continue to be aware of field practice and instrumentation and the associated errors which can arise in data. Some of the secondary validation checks are oriented towards the specific types of errors just mentioned, whilst others are general in nature and lead to identification of spatial inconsistencies in the data. Rainfall poses particular problems for spatial comparisons because of the limited or uneven correlation between stations. When rainfall is convectional in type, it may rain heavily at one location, whilst another only a few miles away may remain dry. Over a month or a monsoon season, such spatial unevenness tends to be smoothed out and aggregated totals are much more closely correlated. Spatial correlation in rainfall depends on type of precipitation, physiographic characteristics of the region, duration (decreases as duration decreases) and distance (decreases as distance increases). Examples of many of the techniques described in this section are given in Hydro- Meteorology Training Module 09 “How to carry out secondary validation of rainfall data” and Training Module 10 “How to correct and complete rainfall data”.
  31. 31. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 25 4.3.2 Comparison with data limits for totals at longer durations Daily rainfall data are aggregated to monthly, seasonal and yearly time intervals for checking if the resulting data series is consistent with the prescribed data limits for such time intervals. This is a useful technique for identifying small systematic or persistent observation errors in rainfall that are not apparent at a daily time interval but which tend to accumulate with time and, therefore, become more visible. As well as upper warning levels or maximum limits, for monsoon months and yearly values, use of lower warning levels can be made to see if certain values are unexpectedly low and thus warrant a closer look. This is a valuable secondary check for SRGs at which there is no sub- daily raingauge for comparison. Suspect aggregated values are flagged and commented appropriately for further validation on the basis of data from nearby stations. 4.3.3 Comparison of rainfall from multiple stations A combination of graphical and tabular displays of hourly, daily, monthly, seasonal and annual rainfall data from multiple stations in a region provides an efficient way of identifying anomalies. Where only two stations are involved in the comparison, the identification of an anomaly does not necessarily indicate which station is at fault. Specific checks to make include: • Do the daily blocks of rain days generally coincide in start day and finish day? • Are there exceptions that are misplaced, starting one day early or late? • Is there a consistent pattern of misfit for a station through the month? • Are there days with no rainfall at a station when (heavy) rainfall has occurred at all neighbouring stations? For multiple rainfall time series plots, select a set of stations within a small area with an expectation of spatial correlation including in the set, if possible, one or more stations which historically have been more reliable. Plot the rainfall series as histograms, preferably in different colours for each station. Side by side plotting of histograms permits comparison on the magnitudes of rainfall at different stations, whilst one above the other plotting makes time shifts easier to detect. This is a valuable secondary check for SRGs at which there is no sub-daily raingauge for comparison. Rainfall occurs in dry and wet spells and SRG-only observers may fail to record the zeros during the dry spells and, hence, lose track of the date when the next rain arrives. When ancillary climate data are available, this may be used to compare with rainfall data e.g. a day with unbroken sunshine in which rain has been reported suggests that rainfall has been reported for the wrong day. However, most comparisons are not so clear cut and the user should be aware that there are a number of possibilities: • Rainfall data only on the wrong day - anomalies between rainfall and climate and between rainfall and neighbouring rainfall • Rainfall and climate data both reported on the wrong day - hence no anomaly between them but discrepancy with neighbouring stations • Rainfall and climate both reported on the correct day - the anomaly was in the occurrence of rainfall e.g. no rainfall at one site but at neighbouring sites. In this case, climate variables are likely to have been shared between neighbouring stations even if rainfall did not occur Unexplained anomalies should initially be followed up by checking the field documents to check for unnoticed mistakes during data entry or primary validation, in which case the data can be corrected accordingly. If necessary, the anomaly should be communicated to the supervising field officer and observer to confirm data and/or rectify problems. Data still regarded as suspect after follow-up checking are flagged and commented appropriately for further validation on the basis of data from nearby stations.
  32. 32. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 26 Figure 4.1 Definition of test and neighbouring stations 4.3.4 Spatial homogeneity testing of rainfall Spatial homogeneity testing of rainfall data is a technique where the observed rainfall at a station station is compared with an estimate of the rainfall at that station, based on the weighted average of rainfall from nearby stations. This approach depends on the degree of spatial consistency of the rainfall, which is primarily based on the actual spatial correlation. This check for spatial consistency can be carried out for various durations of rainfall accumulations. Wherever the difference between the observed and the estimated values exceeds the expected limiting value, such observed values are considered suspect values and flagged for further investigation to ascertain the possible causes of the differences. It is only possible to include a brief description of the technique below, but more detailed information is provided in SW8-OM(II) Chapter 2.6 and Chapter 3.6. Firstly, the nearby stations to be used should be chosen. The stations selected as neighbours should be physically representative of the area in which the “test station” (the station under scrutiny) is situated. The following criteria are used to select the neighbouring stations (Figure 4.1): • The distance between the test and the neighbouring station should be less than a specified maximum correlation distance, say Rmax km • A maximum of 8 neighbouring stations can be considered for interpolation • To reduce the spatial bias in selection, it is appropriate to consider a maximum of only two stations within each quadrant. Secondly, having selected the neighbouring stations, the estimation of the spatially interpolated rainfall is made at the test station, by computing the weighted average of the rainfall observed at neighbouring stations. The estimate of the interpolated value at the test station based on the observations at M neighbouring stations is given as: Where: Pest,j = estimated rainfall at the test station at time j Pi,j = observed rainfall at the neighbour station i at time j
  33. 33. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 27 Di = distance between the test and the neighbouring station i Mbase = number of neighbouring stations taken into account b = power of distance D used for weighting rainfall values at individual station e.g. 2 Thirdly, this estimated rainfall is compared with the observed rainfall at the test station and the difference is considered insignificant if: • The difference between the observed and estimated rainfall at the test station is less than or equal to the admissible absolute difference Xabs • The difference between the observed and estimated rainfall at the test station is less than or equal to a multiplier Xrel of the standard deviation SPest,j of the neighbouring values given by: Where differences between the observed and estimated are unacceptably high, the observed value should be flagged “+” or “-”, depending on whether the observed rainfall is greater or less than the estimated one. Typical rainfall measurement errors show up with specific patterns of “+” and “-”. The limits Xabs and Xrel are chosen by the user and have to be based on the spatial variability of rainfall, and may be altered seasonally. They are normally determined on the basis of experience with the historical data with the objective of flagging a few values (say 2-3%) as suspect values. It is customary to select a reasonably high value of Xabs and a value of Xrel which avoid having to deal with a large number of difference values in the lower range where differences are more likely to occur and have less effect on the overall rainfall total. 4.3.5 Double mass analysis Double mass analysis is a technique to detect a systematic shift, like abrupt or gradual changes, in a rainfall time series, persisting for a considerable period of time. Such inconsistencies can occur for various reasons: • The raingauge might have been installed at different sites in the past • The exposure conditions of the gauge may have undergone a significant change due to the growth of trees or construction of buildings in its proximity • There might have been a change in the instrument, say from 125 mm to 200 mm raingauge • The raingauge may have been faulty for a considerable period etc A note may be available in the station files of the known changes of site and instruments and can be used to corroborate the detection of inconsistencies. The application of double mass analysis to rainfall data is not be possible until a significant amount of historical data is available. The accumulated rainfall at the test station (the station under scrutiny) is compared with another accumulated rainfall series that is expected to be homogeneous. The homogeneous time series can be from an individual reliable station or be an average of reliable time series from neighbouring stations, referred to as the base station(s). Accumulation of rainfall can be made from daily data to monthly or yearly duration. It is only possible to include a brief description of the technique below, but more detailed information is provided in SW8-OM(II) Chapter 2.12 and Chapter 3.5. Firstly, the double mass plot between the accumulated rainfall values in absolute or percent form at test and base stations is drawn and observed for any visible change in its slope. The tabular output giving the ratio between the accumulated rainfall values at test and base stations in absolute and percent form is also obtained. The analysis can be carried for only a part of the years
  34. 34. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 28 or months, if there are some missing rainfall values within the time series (which may themselves indicate some change at the station). Secondly, any visible change in the slope of the double mass plot should be noted. If the data of the test station is homogeneous and consistent with the data of the base station(s), the double mass curve will show a straight line. An abrupt change in the test rainfall series will create a break in the double mass curve, whereas a trend will create a curve. A change in slope is not usually considered significant unless it persists for at least 5 years, and it does not imply that either period is incorrect, simply that it is inconsistent. Furthermore, double mass analysis is based on the assumption that only a part of the rainfall time series under consideration is subject to systematic shift. Where the whole rainfall time series has such a shift, the double mass analysis will fail to detect any inconsistency. Any significant inconsistency that is detected should be investigated further to explore possible causes. If the inconsistency is caused by changed exposure conditions or shift in the station location or systematic instrumental error, the rainfall series should be considered suspect. Thirdly, the suspect part of the rainfall series can be made homogeneous by suitably transforming it before/after the period of shift as required. The earlier part of the record may be adjusted so that that the entire record is consistent with the present and continuing record, or the latter part of the record may be adjusted when the source of the inconsistency is known and has been removed or where the record has been discontinued. Transformation is carried out by multiplying it by a correction factor which is the ratio of the slope of the adjusted mass curve to the slope of the unadjusted mass curve. In the double mass plot shown in Figure 4.2, there is a distinct break at time T1. If the start and end times of the period under consideration are T0 and T2, respectively, then the slopes of the curve before α1 and after α2 the break point can be expressed as: Figure 4.2 Definition sketch for double mass analysis
  35. 35. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 29 In the case that the earlier part between T0 and T1 needs to be corrected for, the correction factor and the corrected observations at the test station can be expressed as: After making such a correction, the double mass curve should be plotted again to check that there is no significant change in the slope of the curve. 4.4 Correction and completion 4.4.1 Overview Completion – the processing of filling in missing values and correcting erroneous values – is done as a continuous process with primary and secondary validation. Although the HIS Manual SW separates correction and completion in SW8-OM(II) Chapter 3 from secondary validation in SW8- OM(II) Chapter 2, and from primary validation in SW8-OM(I) Chapter 5, there is substantial overlap between the techniques used. In this Handbook, some correction and completion techniques have been included in the appropriate parts of Sections 4.2 and 4.3, and others are described below. Examples of many of the techniques described, which should be carried out by experienced staff with appropriate training, are given in Hydro-Meteorology Training Module 10 “How to correct and complete rainfall data”. The majority of secondary validation, and therefore the majority of correction and completion, is carried out by State DPCs to take advantage of the information available from a larger area. For the Hydrology Project, correction and completion (see Table 3.1) should be completed by the end of the following month (e.g. for June data by 31st July). Some secondary validation, correction and completion will not be possible until the end of the hydrological year when the entire year’s data can be reviewed in a long-term context, so data should be regarded as provisional approved data until then (e.g. for June data by the end of the hydrological year plus 3 months), after which data should be formally approved and made available for dissemination to external users. Procedures for correction and completion depend on the type of error and the availability of suitable source records with which to estimate. It should be recognised that values estimated from other gauges are inherently less reliable than values properly measured. There will be circumstances where no suitable neighbouring observations or stations are available, such that missing values should be left as -999 and incorrect values should be set to -999, and suspect original values should be given the benefit of the doubt and retained in the record with an appropriate flag. 4.4.2 Correcting missing and erroneous data Missing values (-999 or incorrect zeros) may be the result of the observer failing to make an observation, failing to enter the observation in the record sheet, or entering the observation incorrectly. For rainy periods, missed values will be anomalous in the multiple station tabulation and plot and will be indicated by a series of “-” departures in the spatial homogeneity test. Where such missed entries are confidently identified, the missed values should be replaced by the estimates derived from neighbouring stations by spatial interpolation. This is also the approach for values confidently identified as erroneous. Where there is some doubt as to the interpretation, the
  36. 36. Hydrological Information System May 2014 HP II Last Updated: 19/05/2014 05:01 Filename: MET Handbook.docx Page 30 value should be left unchanged but flagged as suspect. There are three analytical procedures for estimating rainfall by spatial interpolation: • Arithmetic average – applied if the average annual rainfall of the station under consideration is within 10% of the average annual rainfall at the neighbouring stations. The missing or erroneous rainfall at the station under consideration is estimated as the simple average of neighbouring stations. Thus, if the estimate for the missing or erroneous rainfall at the station under consideration is Ptest and the rainfall at M adjoining stations is Pbase,i (i = 1 to M), then: Usually, averaging of three or more neighbouring stations is considered to give a satisfactory estimate. • Normal ratio – applied if the average annual rainfall of the station under consideration differs from the average annual rainfall at the neighbouring stations by more than 10%. The erroneous or missing rainfall at the station under consideration is estimated as the weighted average of the data at the neighbouring stations. The rainfall at each of the neighbouring stations is weighted by the ratio of the average annual rainfall at the station under consideration and average annual rainfall of the neighbouring station. The rainfall for the missing or erroneous value at the station under consideration is estimated as: Where: Ntest = annual average rainfall at the station under consideration Nbase,i = annual average rainfall at the adjoining stations (for i = 1 to M) A minimum of three neighbouring stations should generally be used for obtaining good estimates using the normal ratio method. • Distance power – this is the approach used for the spatial homogeneity test in Section 4.3.4 of this Handbook, and weights neighbouring stations on the basis of their distance from the station under consideration, on the assumption that closer stations are better correlated than those further away and that, beyond a certain distance, they are insufficiently correlated to be of use. Spatial interpolation is made by weighing the adjoining station rainfall as inversely proportional to some power (e.g. 2) of the distances from the station under consideration. However, due to prevailing wind conditions or orographic effects, spatial heterogeneity may be present in which case, normalised values rather than actual values should be used in the interpolation. The observed rainfall values at the neighbouring stations are multiplied by the ratio of the average annual rainfall at the test station and the average annual rainfall at the neighbouring stations: Where: Pcorr,i,j = for corrected rainfall value at the neighbouring station i at time j Ntest = annual average rainfall at the station under consideration Nbase,i = annual average rainfall at the neighbouring stations (for i = 1 to Mbase) 4.4.3 Correcting accumulated data Should an observer miss one or more readings, he/she may make one of three choices for the missed period of record:

×