SlideShare a Scribd company logo
1 of 36
Download to read offline
QC Challenges of Collecting
Time-Continuous Water
Quality Data:
PacIOOS Water Quality Buoy (WQB) Data QC
by
Michael Tomlinson & Eric De Carlo
Handout &Outline
 Handout contains most data products in presentation
 Overview of Pacific Islands Ocean Observing System
 PacIOOS water quality components
 Presentation focuses on the NOAA/PMEL MapCO2 /
PacIOOS specifically Water Quality Buoy data QC
 Manual data processing & QC (different time scales)
 Annual (and beyond) QC & data adjustment with
Aquarius® software
 Independent data (& estimate) validation
 Final data release
 What have we learned from 5 years of monitoring?
IOOS – National Ocean Observing
PacIOOS
PacIOOS
What does
PacIOOS do?
 Data Collection & QC
 Data Aggregation
 Data Visualization
 Real-time Warnings
 Provides Research Data
 Outreach/Communication
Examples of Applications
Understanding Ocean Acidification
Assessing the effects of storms on coastal
water quality (e.g., Tomlinson et al., 2011)
Water quality monitoring & sounding “all
clear” after tsunamis (e.g., Fiedler et al., 2014)
Impacts of groundwater / surface water on
coastal marine ecosystems
 NOAA/PMEL MapCO2/
PacIOOS WQBs (T, S, O2, CO2,
pH, nitrate, turbidity,
chlorophyll)
 Nearshore Sensors (T, S,
turbidity, chlorophyll)
 Kilo Nalu Observatory (tides,
waves, currents, acoustic
backscatter, T)
 Autonomous Underwater
Vehicle (bathy, currents, T, S,
chlorophyll, optical
backscatter)
 Synoptic manual event-driven
sampling
 NOAA/NWS Met Stations &
Rain Gages
 USGS Streamflow Gages
 NOAA/NOS Tide Gages & Met
Stations
 HIDoH Beach Monitoring
 NOAA/NDBC Open Ocean
Data Buoys
 Satellite Imagery
PacIOOS Water Quality Components
Water Quality Component Locations
NOAA/PMEL MapCO2/PacIOOS WQB Sensors
Sensor Variable
SBE16plus
V2 SEACAT®
Temperature
Salinity
Pressure
SBE63 DO® Diss. Oxygen
WET Labs
FLNTUS®
Chlorophyll
Turbidity
ISUS-V3® Nitrate
SeaFET® pH
LI-COR CO2® CO2
Data Processing & QC Flowchart
Figure 1
Twice-Daily Checks
Twice-Daily Checks (continued)
Gaps in the WQB record on
the website that did not
exist (see graph below);
caused by a problem with
the Data Turbine
Manual Processing (every 3 days)
Monthly QC Check
MathCAD® Check of Spreadsheet
Coefficient Check
Check Spreadsheet Calculation
Monthly Time-Series Plot
WQB Data QC 1 Year & Beyond
Aquarius "whiteboard" used to QC and adjust, as
necessary, data from the PacIOOS WQBs. In addition to
the WQB data, other data are used as surrogates to
help QC the data and guide any necessary
adjustments. The raw, adjusted, surrogate, and discrete
(C&CH) data are plotted as time-series.
Figure 2
Partial Aquarius® Whiteboard (close-up)
Rare Adjustment to Salinity Data
Raw Data
Adjusted Data
Note: Adjusted data are always flagged as ESTIMATES
Figure 3
Not So Rare Adjustment of
SBE43® Dissolved Oxygen (DO) Sensor
Figure 4
WET Labs FLNTUS® Chlorophyll &
Turbidity Sensor – Ultimate Challenge
Figure 5
Great when the sensor is working …
Figure 6
… a real challenge when it isn’t working
Figure 7
Final Adjusted Turbidity & Surrogates
Figure 8
Chlorophyll had its own challenges …
Figure 9
… but adjustable trim correction …
Figure 10
… salvaged some of the data!
Figure 11
Problems Evident After 1+ Years
(SBE43® Membrane DO Sensor)
Sensor began to drift immediately after deployment and continued
throughout the entire deployment period, regardless of the time of
year. SBE conceded there was a problem but cannot explain it.
Figure 12
Correcting the SBE43® DO Problem
Figure 13
Independent Data QC Check
C&CH NPDES
Receiving Water
Monitoring Station
Figure 14
Raw &
Adj.
WQB
Data
and
C&CH
NPDES
Data
Figure 15
Figure 16
Master PacIOOS WQB Spreadsheets
PacIOOS Water Quality Webpage
What have we learned in 5 years?
 QC must be conducted at different time scales
 Need one extra suite of sensors for annual servicing
 Some sensors (e.g., SBE43® DO membrane sensor)
do not work well in shallow, subtropical water
 Although expensive, need another suite of sensors with
comparable precision & accuracy for servicing checks
 Data QC is not part-time job nor can it be automated …
 … unless we emulate the National Data Buoy Center &
have redundant sensors (very expensive)
 Routine monitoring provides valuable research data,
e.g., tsunami water quality effects (Fiedler et al., 2014)
Questions & Contact Information
www.pacioos.org
Michael Tomlinson
UHM Oceanography
Flagstaff, AZ
mtomlins@hawaii.edu
928-266-2236 (cell)

More Related Content

Similar to PacIOOS WQB Data QC

Datalogger error introductory course
Datalogger error introductory courseDatalogger error introductory course
Datalogger error introductory courseHydroG Resources
 
Long-term stability of a moored optical oxygen sensor in an estuary
Long-term stability of a moored optical oxygen sensor in an estuaryLong-term stability of a moored optical oxygen sensor in an estuary
Long-term stability of a moored optical oxygen sensor in an estuarySeaBirdScientific
 
Academia: Richard Lawford, Morgan State University, 16th January UN Water Zar...
Academia: Richard Lawford, Morgan State University, 16th January UN Water Zar...Academia: Richard Lawford, Morgan State University, 16th January UN Water Zar...
Academia: Richard Lawford, Morgan State University, 16th January UN Water Zar...water-decade
 
C1.06: US IOOS: Eyes on the Ocean - Zdenka Willis
C1.06: US IOOS: Eyes on the Ocean - Zdenka WillisC1.06: US IOOS: Eyes on the Ocean - Zdenka Willis
C1.06: US IOOS: Eyes on the Ocean - Zdenka WillisBlue Planet Symposium
 
The use of imagery in monitoring existing sub surface structures
The use of imagery in monitoring existing sub surface structuresThe use of imagery in monitoring existing sub surface structures
The use of imagery in monitoring existing sub surface structuresPeter McCready
 
Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
 Download-manuals-surface water-manual-sw-volume6designmanualwqsampling Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
Download-manuals-surface water-manual-sw-volume6designmanualwqsamplinghydrologyproject001
 
Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
 Download-manuals-surface water-manual-sw-volume6designmanualwqsampling Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
Download-manuals-surface water-manual-sw-volume6designmanualwqsamplinghydrologywebsite1
 
Directions OGC CHISP-1 Webinar Slides
Directions OGC CHISP-1 Webinar SlidesDirections OGC CHISP-1 Webinar Slides
Directions OGC CHISP-1 Webinar SlidesAlex Joseph
 
Sec Process Control Past Present and Future2222
Sec Process Control Past Present and Future2222Sec Process Control Past Present and Future2222
Sec Process Control Past Present and Future2222Roger Colee
 
Download-manuals-ground water-manual-gw-volume8operationmanualdataprocessing...
 Download-manuals-ground water-manual-gw-volume8operationmanualdataprocessing... Download-manuals-ground water-manual-gw-volume8operationmanualdataprocessing...
Download-manuals-ground water-manual-gw-volume8operationmanualdataprocessing...hydrologyproject001
 
DSD-INT 2015 - Satellite based near-real time information services for aquati...
DSD-INT 2015 - Satellite based near-real time information services for aquati...DSD-INT 2015 - Satellite based near-real time information services for aquati...
DSD-INT 2015 - Satellite based near-real time information services for aquati...Deltares
 
C1.01: GOOS: an essential collaborative system enabling societal benefit - Jo...
C1.01: GOOS: an essential collaborative system enabling societal benefit - Jo...C1.01: GOOS: an essential collaborative system enabling societal benefit - Jo...
C1.01: GOOS: an essential collaborative system enabling societal benefit - Jo...Blue Planet Symposium
 
Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
 Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys... Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...hydrologyproject001
 
Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
 Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys... Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...hydrologywebsite1
 
Tudor Dragea ICONE 23 Final Paper
Tudor Dragea ICONE 23 Final Paper Tudor Dragea ICONE 23 Final Paper
Tudor Dragea ICONE 23 Final Paper Tudor Dragea
 
Long Term Monitoring Data 2013
Long Term Monitoring Data 2013Long Term Monitoring Data 2013
Long Term Monitoring Data 2013James Peale
 
Download-manuals-surface water-waterlevel-21howtomakedataentryforwaterleveldata
 Download-manuals-surface water-waterlevel-21howtomakedataentryforwaterleveldata Download-manuals-surface water-waterlevel-21howtomakedataentryforwaterleveldata
Download-manuals-surface water-waterlevel-21howtomakedataentryforwaterleveldatahydrologyproject001
 
DSD-INT 2016 Chautauqua; the Alberta groundwater information system using Del...
DSD-INT 2016 Chautauqua; the Alberta groundwater information system using Del...DSD-INT 2016 Chautauqua; the Alberta groundwater information system using Del...
DSD-INT 2016 Chautauqua; the Alberta groundwater information system using Del...Deltares
 

Similar to PacIOOS WQB Data QC (20)

Datalogger error introductory course
Datalogger error introductory courseDatalogger error introductory course
Datalogger error introductory course
 
Long-term stability of a moored optical oxygen sensor in an estuary
Long-term stability of a moored optical oxygen sensor in an estuaryLong-term stability of a moored optical oxygen sensor in an estuary
Long-term stability of a moored optical oxygen sensor in an estuary
 
Academia: Richard Lawford, Morgan State University, 16th January UN Water Zar...
Academia: Richard Lawford, Morgan State University, 16th January UN Water Zar...Academia: Richard Lawford, Morgan State University, 16th January UN Water Zar...
Academia: Richard Lawford, Morgan State University, 16th January UN Water Zar...
 
C1.06: US IOOS: Eyes on the Ocean - Zdenka Willis
C1.06: US IOOS: Eyes on the Ocean - Zdenka WillisC1.06: US IOOS: Eyes on the Ocean - Zdenka Willis
C1.06: US IOOS: Eyes on the Ocean - Zdenka Willis
 
The use of imagery in monitoring existing sub surface structures
The use of imagery in monitoring existing sub surface structuresThe use of imagery in monitoring existing sub surface structures
The use of imagery in monitoring existing sub surface structures
 
Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
 Download-manuals-surface water-manual-sw-volume6designmanualwqsampling Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
 
Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
 Download-manuals-surface water-manual-sw-volume6designmanualwqsampling Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
Download-manuals-surface water-manual-sw-volume6designmanualwqsampling
 
Directions OGC CHISP-1 Webinar Slides
Directions OGC CHISP-1 Webinar SlidesDirections OGC CHISP-1 Webinar Slides
Directions OGC CHISP-1 Webinar Slides
 
Sec Process Control Past Present and Future2222
Sec Process Control Past Present and Future2222Sec Process Control Past Present and Future2222
Sec Process Control Past Present and Future2222
 
Download-manuals-ground water-manual-gw-volume8operationmanualdataprocessing...
 Download-manuals-ground water-manual-gw-volume8operationmanualdataprocessing... Download-manuals-ground water-manual-gw-volume8operationmanualdataprocessing...
Download-manuals-ground water-manual-gw-volume8operationmanualdataprocessing...
 
DSD-INT 2015 - Satellite based near-real time information services for aquati...
DSD-INT 2015 - Satellite based near-real time information services for aquati...DSD-INT 2015 - Satellite based near-real time information services for aquati...
DSD-INT 2015 - Satellite based near-real time information services for aquati...
 
Mercator Ocean newsletter 43
Mercator Ocean newsletter 43Mercator Ocean newsletter 43
Mercator Ocean newsletter 43
 
C1.01: GOOS: an essential collaborative system enabling societal benefit - Jo...
C1.01: GOOS: an essential collaborative system enabling societal benefit - Jo...C1.01: GOOS: an essential collaborative system enabling societal benefit - Jo...
C1.01: GOOS: an essential collaborative system enabling societal benefit - Jo...
 
Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
 Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys... Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
 
Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
 Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys... Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
Download-manuals-water quality-wq-manuals-gwvolume6designmanualwaterqualitys...
 
Tudor Dragea ICONE 23 Final Paper
Tudor Dragea ICONE 23 Final Paper Tudor Dragea ICONE 23 Final Paper
Tudor Dragea ICONE 23 Final Paper
 
Long Term Monitoring Data 2013
Long Term Monitoring Data 2013Long Term Monitoring Data 2013
Long Term Monitoring Data 2013
 
Download-manuals-surface water-waterlevel-21howtomakedataentryforwaterleveldata
 Download-manuals-surface water-waterlevel-21howtomakedataentryforwaterleveldata Download-manuals-surface water-waterlevel-21howtomakedataentryforwaterleveldata
Download-manuals-surface water-waterlevel-21howtomakedataentryforwaterleveldata
 
DSD-INT 2016 Chautauqua; the Alberta groundwater information system using Del...
DSD-INT 2016 Chautauqua; the Alberta groundwater information system using Del...DSD-INT 2016 Chautauqua; the Alberta groundwater information system using Del...
DSD-INT 2016 Chautauqua; the Alberta groundwater information system using Del...
 
QUOVADIS_NUM3_OND_2010
QUOVADIS_NUM3_OND_2010QUOVADIS_NUM3_OND_2010
QUOVADIS_NUM3_OND_2010
 

More from MSTomlinson

Tomlinson et al (2016) - sediment & biota
Tomlinson et al (2016) - sediment & biotaTomlinson et al (2016) - sediment & biota
Tomlinson et al (2016) - sediment & biotaMSTomlinson
 
Oceanography in the Private Sector (M Tomlinson)
Oceanography in the Private Sector (M Tomlinson)Oceanography in the Private Sector (M Tomlinson)
Oceanography in the Private Sector (M Tomlinson)MSTomlinson
 
MST_ProjExpSumm (as of 2014)
MST_ProjExpSumm (as of 2014)MST_ProjExpSumm (as of 2014)
MST_ProjExpSumm (as of 2014)MSTomlinson
 
MST Project Management List (2014)
MST Project Management List (2014)MST Project Management List (2014)
MST Project Management List (2014)MSTomlinson
 
Monitoring in the Ahupuaa-s
Monitoring in the Ahupuaa-sMonitoring in the Ahupuaa-s
Monitoring in the Ahupuaa-sMSTomlinson
 
Oahu Soil & Sediment As
Oahu Soil & Sediment AsOahu Soil & Sediment As
Oahu Soil & Sediment AsMSTomlinson
 
MS Tomlinson Defense
MS Tomlinson DefenseMS Tomlinson Defense
MS Tomlinson DefenseMSTomlinson
 
MS Tomlinson Thesis 2004-s
MS Tomlinson Thesis 2004-sMS Tomlinson Thesis 2004-s
MS Tomlinson Thesis 2004-sMSTomlinson
 
Statistical Analysis of Left-Censored Geochemical Data
Statistical Analysis of Left-Censored Geochemical DataStatistical Analysis of Left-Censored Geochemical Data
Statistical Analysis of Left-Censored Geochemical DataMSTomlinson
 

More from MSTomlinson (9)

Tomlinson et al (2016) - sediment & biota
Tomlinson et al (2016) - sediment & biotaTomlinson et al (2016) - sediment & biota
Tomlinson et al (2016) - sediment & biota
 
Oceanography in the Private Sector (M Tomlinson)
Oceanography in the Private Sector (M Tomlinson)Oceanography in the Private Sector (M Tomlinson)
Oceanography in the Private Sector (M Tomlinson)
 
MST_ProjExpSumm (as of 2014)
MST_ProjExpSumm (as of 2014)MST_ProjExpSumm (as of 2014)
MST_ProjExpSumm (as of 2014)
 
MST Project Management List (2014)
MST Project Management List (2014)MST Project Management List (2014)
MST Project Management List (2014)
 
Monitoring in the Ahupuaa-s
Monitoring in the Ahupuaa-sMonitoring in the Ahupuaa-s
Monitoring in the Ahupuaa-s
 
Oahu Soil & Sediment As
Oahu Soil & Sediment AsOahu Soil & Sediment As
Oahu Soil & Sediment As
 
MS Tomlinson Defense
MS Tomlinson DefenseMS Tomlinson Defense
MS Tomlinson Defense
 
MS Tomlinson Thesis 2004-s
MS Tomlinson Thesis 2004-sMS Tomlinson Thesis 2004-s
MS Tomlinson Thesis 2004-s
 
Statistical Analysis of Left-Censored Geochemical Data
Statistical Analysis of Left-Censored Geochemical DataStatistical Analysis of Left-Censored Geochemical Data
Statistical Analysis of Left-Censored Geochemical Data
 

PacIOOS WQB Data QC

  • 1. QC Challenges of Collecting Time-Continuous Water Quality Data: PacIOOS Water Quality Buoy (WQB) Data QC by Michael Tomlinson & Eric De Carlo
  • 2. Handout &Outline  Handout contains most data products in presentation  Overview of Pacific Islands Ocean Observing System  PacIOOS water quality components  Presentation focuses on the NOAA/PMEL MapCO2 / PacIOOS specifically Water Quality Buoy data QC  Manual data processing & QC (different time scales)  Annual (and beyond) QC & data adjustment with Aquarius® software  Independent data (& estimate) validation  Final data release  What have we learned from 5 years of monitoring?
  • 3. IOOS – National Ocean Observing PacIOOS
  • 5. What does PacIOOS do?  Data Collection & QC  Data Aggregation  Data Visualization  Real-time Warnings  Provides Research Data  Outreach/Communication
  • 6. Examples of Applications Understanding Ocean Acidification Assessing the effects of storms on coastal water quality (e.g., Tomlinson et al., 2011) Water quality monitoring & sounding “all clear” after tsunamis (e.g., Fiedler et al., 2014) Impacts of groundwater / surface water on coastal marine ecosystems
  • 7.  NOAA/PMEL MapCO2/ PacIOOS WQBs (T, S, O2, CO2, pH, nitrate, turbidity, chlorophyll)  Nearshore Sensors (T, S, turbidity, chlorophyll)  Kilo Nalu Observatory (tides, waves, currents, acoustic backscatter, T)  Autonomous Underwater Vehicle (bathy, currents, T, S, chlorophyll, optical backscatter)  Synoptic manual event-driven sampling  NOAA/NWS Met Stations & Rain Gages  USGS Streamflow Gages  NOAA/NOS Tide Gages & Met Stations  HIDoH Beach Monitoring  NOAA/NDBC Open Ocean Data Buoys  Satellite Imagery PacIOOS Water Quality Components
  • 9. NOAA/PMEL MapCO2/PacIOOS WQB Sensors Sensor Variable SBE16plus V2 SEACAT® Temperature Salinity Pressure SBE63 DO® Diss. Oxygen WET Labs FLNTUS® Chlorophyll Turbidity ISUS-V3® Nitrate SeaFET® pH LI-COR CO2® CO2
  • 10. Data Processing & QC Flowchart Figure 1
  • 12. Twice-Daily Checks (continued) Gaps in the WQB record on the website that did not exist (see graph below); caused by a problem with the Data Turbine
  • 15. MathCAD® Check of Spreadsheet Coefficient Check Check Spreadsheet Calculation
  • 17. WQB Data QC 1 Year & Beyond Aquarius "whiteboard" used to QC and adjust, as necessary, data from the PacIOOS WQBs. In addition to the WQB data, other data are used as surrogates to help QC the data and guide any necessary adjustments. The raw, adjusted, surrogate, and discrete (C&CH) data are plotted as time-series. Figure 2
  • 19. Rare Adjustment to Salinity Data Raw Data Adjusted Data Note: Adjusted data are always flagged as ESTIMATES Figure 3
  • 20. Not So Rare Adjustment of SBE43® Dissolved Oxygen (DO) Sensor Figure 4
  • 21. WET Labs FLNTUS® Chlorophyll & Turbidity Sensor – Ultimate Challenge Figure 5
  • 22. Great when the sensor is working … Figure 6
  • 23. … a real challenge when it isn’t working Figure 7
  • 24. Final Adjusted Turbidity & Surrogates Figure 8
  • 25. Chlorophyll had its own challenges … Figure 9
  • 26. … but adjustable trim correction … Figure 10
  • 27. … salvaged some of the data! Figure 11
  • 28. Problems Evident After 1+ Years (SBE43® Membrane DO Sensor) Sensor began to drift immediately after deployment and continued throughout the entire deployment period, regardless of the time of year. SBE conceded there was a problem but cannot explain it. Figure 12
  • 29. Correcting the SBE43® DO Problem Figure 13
  • 30. Independent Data QC Check C&CH NPDES Receiving Water Monitoring Station Figure 14
  • 33. Master PacIOOS WQB Spreadsheets
  • 35. What have we learned in 5 years?  QC must be conducted at different time scales  Need one extra suite of sensors for annual servicing  Some sensors (e.g., SBE43® DO membrane sensor) do not work well in shallow, subtropical water  Although expensive, need another suite of sensors with comparable precision & accuracy for servicing checks  Data QC is not part-time job nor can it be automated …  … unless we emulate the National Data Buoy Center & have redundant sensors (very expensive)  Routine monitoring provides valuable research data, e.g., tsunami water quality effects (Fiedler et al., 2014)
  • 36. Questions & Contact Information www.pacioos.org Michael Tomlinson UHM Oceanography Flagstaff, AZ mtomlins@hawaii.edu 928-266-2236 (cell)