Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Applying Electronic Data Checking Tools To Environmental Data


Published on

Presentation given at the 23rd Annual National Conference on Managing Environmental Quality Systems, April 15, 2004.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Applying Electronic Data Checking Tools To Environmental Data

  1. 1. Applying Electronic Data Checking Tools to Environmental Data Thursday, April 15, 2004 9:30 AM—10:00 AM 23rd Annual National Conference on Managing Environmental Quality Systems Tampa Marriott Waterside Hotel and Marina, Tampa, Florida Introduction The rate at which environmental data is being generated today and the importance of managing that data in real time with reduced resources is driving a renewed look by all participants in the value of using electronic tools in the collection of environmental data. This presentation will introduce the concepts of electronic data download (EDD) file definitions, data checking, conditional processing, and automated data transmittals. A leading edge data PC processing tool will be introduced and opportunity for hands on experience will be provided. Further automation of the data path for data importing and subsequent actions on receipt such as action level criteria checking; push reporting; and automated emails will also be explored. A new generation data verification tool will also be presented that assists data validators in finding the “needle in the haystack” errors that require attention to ensure the data received meets the project’s intended data quality objectives. Additional opportunity to review the products and procedures presented will be afforded following the presentation, or upon request at Key Concepts Presented: Importance of Data Quality o First address Data Quality, then Data Usability o Data Usability without Data Quality is ‘Garbage In - Garbage Out’. o Bad data can make beautiful graphs! But, Good data leads to Good Decisions. Existing Challenges o Variations in formats o Inconsistent values o Missing content o Errors in content o Verification at end of the process leaves little ability to make corrections – TQM? o Much Manual effort required for success Solutions o Define data formats for each data type o Establish consistent valid values o Check EDDs for compliance and report errors for correction – at generation o Verify EDD at receipt o Define data package for automated processing
  2. 2. EDP – for Data Submitters o A .Net product, using XML and easily deployed. o Data producers can check and correct EDDs. o Custom Checking scripts may be included with formats – special checking can be configured for: Required fields and Reference Values Conditional checking (if this field is X, then that field should be Y) Other special or custom required data reviews QC recovery checking Check multiple EDD format tables simultaneously EQuIS Data Processor (EDP) - Additional Workflow Values o Allows end user to use XML to create new EDDs. o User Login/identification, and Audit trail capable. o Generate error report including row number, error, and value. o Screen data against required limits, with different regulatory limits per location. o Upload EDD from user, via email, ftp, or portal. o Web Enabled to automate EDD processing for high volume operations. o Optionally upload data into defined database tables. Data Entry - DATA ENTRY FORMS for manual data entry are useful for historical data conversions and for field data entry. o This XML tool for Laptops or Tablet PCs enables a controlled dialog to create the import EDD files in a familiar layout for field staff. EDP Online o Labs and Data Providers can submit data online or via email. o Pass or Fail results (with Error Logs, if any) are emailed back to data provider. o Automation By Email or Online Verify Submitter Auto process Package Maintain Audit Trail Email EDD messages Load accepted data Initiate Push processing DQM – Data Qualification Module o Automate the process of Data Verification Checks. o Reduce staff time checking good data. o Find the Needle-in-a-haystack errors. o Focus experts on resolving problems. o Enable higher percentage checks – up to 100% for key data items. DQM Procedures o Review Data by SDG
  3. 3. Set Up Criteria by Method/Matrix o Enter Method and Matrix Specific Holding Times, Qualifiers, etc… o Select Checks and Qualifier Rules o Run Checks, Review Error Log o Commit Results to the Database o Generate Report Tables o Electronic Checks o DQM Benefits for Data Verification o Produce better quality and more reliable data. o Implement discovery and correction procedures at data submittal – TQM. o Enable checking of a higher percentage of data. o Enable automation of routine processes. o Better, Faster, Cheaper overall process. Questions