Electronic Data Management and Workflow


Published on

EarthSoft LADEQ Presentation given by Arcadis at the 2009 EQuIS User Group Meetup

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Electronic Data Management and Workflow

  1. 1. Electronic Data Management and Workflow Imagine the result
  2. 2. Introduction Jane Kennedy Project Chemistry & Data Quality ARCADIS U.S., Inc. New Orleans
  3. 3. EDD Management and Data Workflow Overview • ARCADIS data management approach • Planning & setup • Data acquisition • Laboratory EDD prep and submittal • EDD receipt and review • Data distribution • Questions
  4. 4. Electronic Data Management • Why manage data electronically? • Consistency • Confidence • Efficient access to information
  5. 5. ARCADIS Approach to Data Management PLANNING ACQUISITION MANAGEMENT VALIDATION REPORTING Plan data  Acquire data  Manage data  Evaluate data  Provide  acquisition,   efficiently with tools  quality  streamlined  management ,  appropriate  compliance  reporting  quality, and  to output  with project  and data  reporting  systems requirements plan accessibility  to users Increase productivity of technical staff
  6. 6. ARCADIS Data Management Systems • EQuIS 3.2 • Desktop or server • EQuIS 5 • Enterprise system • Automation of EDD management • Internally developed system – Access platform • Server • Microsoft Excel • Desktop
  7. 7. EQuIS 5 Deployment • Involved variety of disciplines and stakeholders in selection • Data managers • Corporate IT • GIS personnel • Project teams • Senior management • Clients • Evaluated overall system applications
  8. 8. Planning A data acquisition and management strategy defines: •Coordination of appropriate resources •Project quality assurance process •Data deliverables •Communication pathways to ensure data usability and accessibility
  9. 9. Stakeholder Participation •Field Data •Boring Logs Database • Project staff •Lab Data •Maps/Figures • Provide information to •3rd Party Info •Data Tables labs and DMs • Data managers • Receipt, import, query and export • Data visualization and project Team • Communicate data export content requirements to DMs
  10. 10. Project Planning Documents • Work Plan, SAP, FSP, QAPP • Document performance requirements • Summarize project activities • Define project goals • Establish data quality objectives • Permits • Corrective action goals • Risk standards (RECAP)
  11. 11. Data Management Plan Data Management Plan • Based on project planning documents • Define personnel responsibilities • Set up work flow • Data acquisition strategy • Establish project nomenclature • Create reference values • Detail storage and archive
  12. 12. Sample Collection Planning • Establish location and sample nomenclature • Identify locations • Monitoring wells • Soil depths • Sample matrix • Trip blanks, equipment blanks • Field duplicates • Provide information to field team and data manager prior to sampling
  13. 13. Database Set Up • Prepare chemistry and geology database • Use applicable project nomenclature • Acquire geographic coordinates • Provide project specific criteria to data manager • Permit limits • Screening standards/RECAP limits • Corrective action goals • Identify export requirements • Trend plots, contours, charts • GIS or CAD
  14. 14. Field Data Acquisition • Establish field data requirements • Well construction • Geologic information • Water quality parameters • Water levels • Identify data acquisition format • Manual entry onto forms with transfer later • Electronic acquisition with nightly upload • Geographic coordinates • Survey or GPS
  15. 15. Uploading Field Data • Routes of data transfer • Who has responsibilities? • What format? • Quality Control Review prior to submittal to Data Manager • Geographic coordinates • Get them to the Data Manager in a timely manner
  16. 16. Chain Of Custody Documentation • Complete COC with as many EDD expectations as possible • Sample ID • Matrix • Sample type • Samples to be used for site specific QC
  17. 17. Communicate Project Requirements to Laboratory • Laboratory is a partner in the project success • Communication prior to sample collection is crucial • Data quality requirements • Performance expectations • Deliverables • Communication tree
  18. 18. What the Laboratory Needs • List of samples and performance criteria • Sample nomenclature • Select EDD Format prior to sample submission • DEQ format • Consultant format • Send project reference values • Confirm lab has programming completed to generate required EDD
  19. 19. Laboratory EDD Preparation • Understand the requirements • Spend the time to develop the 4-file EDD NOW • Minimize manual entry • Report and EDD MUST match • Rounding routines • Manual data entry peer review • Data Checker (EDP) = zero defects • EDD requires specific naming convention • Follow the rules or it will get kicked back
  20. 20. Reference Values Missing • Contact client for direction • Don’t make anything up • Review reference values for other options • Get email confirmation of directions • Include additions changes in email submission of EDD
  21. 21. Potential EDD Pitfalls Sample Table • Confirm sample ID and handwriting interpretations • Do not add any suffixes to the sample ID unless directed by consultant • Sample Delivery Group (report number) must be populated • Lab may be required to populate start and end depth for soils • Sample receipt date must be populated
  22. 22. Potential EDD Pitfalls Sample Table (continued) • Sample type is critical because some samples require listing parent samples • Field duplicates – lab may need to use sample type of “N” to clear checker • MB = Material Blank not method blank (LB)
  23. 23. Potential EDD Pitfalls Test Table • Subsample amounts must be populated • Lab name must be populated • Sample dates and times must match sample file • Percent moisture cannot be null for solids • Analysis type must be appropriate for dilutions, re-analyses • Understand the use of T, D, N for total and dissolved field • Caution with methods where multiple parameters reported (e.g. Method 300 or 352.3) • Time format can cause problems
  24. 24. Potential Pitfalls Result Table • Do not add suffixes to the CAS number • Only 1 result is reportable = Yes (multiple dilutions) • If the detect flag is yes, the result value cannot be null • Units fields cannot be null for populated fields requiring a definition of value units • Subsample amount must be populated • Quality control data must be included in the appropriate field
  25. 25. Additional Laboratory Challenges • Can’t load project reference values • Data Checker continues to show errors • Do NOT try to re-write the export every time you have samples • Make sure to save the export to laboratory system • LIMS changes - CAUTION
  26. 26. EDD Submission • Data checker (EDP) = zero defects • Document problem resolution • Follow EDD naming convention • Email EDD to appropriate venue(s)
  27. 27. PDA Data Receipt: • Variety of information received in Electronic Data Deliverable (EDD) for import into the Field EDD database Lab Lab EDD EDP Submitter Notice Email Web FTP Manager Notice DATABASE
  28. 28. Consultant EDD Management • EDDs received by project Data Manager data manager or via direct upload system • EDP confirms •Field Data •Boring Logs completeness and •Lab Data Database •Maps/Figures compliance (3.2 v 5) • Do not correct the EDD •3rd Party Info •Data Tables • If it is not compliant, return to lab
  29. 29. EDD Upload to Project Database • Manage EDD upload schedule • Group work in batches • Tracking is critical • Report and EDD may not arrive at same time • Verification / validation of conformance
  30. 30. Consultant Database Updates • Verify sample IDs • Laboratory data flag definitions • Run update queries to move qualifiers • Add location codes to link samples • Field information • Field parameters • Parent samples for field dups • Perform project specific queries
  31. 31. Confirmation of Content • Export data to crosstab format • Review a percentage of data against lab reports • Define percentage in project plan process • If deficiencies are identified return to lab for correction • Rounding Routines • Significant Figures • Data Flags
  32. 32. Track Changes to the Database • Identify fields in the database to document changes by data managers • Historical data source • Laboratory Flagging changes • Special data qualifier definitions • Validated, Verified, or Neither • Validator comments/reason for qualification • Updates of project information
  33. 33. Electronic Data Quality Screening • Screening tools are available • Critical to have appropriate QC information • Populate data fields appropriately • Understand the limitations • Qualified personnel review of screening tool reports
  34. 34. ‘My’ Format AND LEADMS • Create LEADMS EDD export from data system • If managing 2 EDDs (client specific and LEADMS) • Ensure any changes in the desktop version are transferred to LEADMS • Perform confirmation QC to verify
  35. 35. Exports to Data Users Tabular data, Crosstabs Charts, Graphs Email ‘Pretty’ Reports Web FTP Exports for data analysis and visualization DATABASE
  36. 36. Data Exporting Data Manager • Quality in = Quality out •Field Data •Boring Logs • Exports can be automated, scheduled or team Database •Lab Data •Maps/Figures coordinated •3rd Party Info •Data Tables • Provide information to the DM to yield efficiency
  37. 37. Data Export Automation • Database monitors incoming data and generate reports based on arrival of: – New data • Anytime there is new data for facility 123, build Report A and email it to me and all my group. – New detections • Anytime there is new Arsenic data greater than 15 ppb for facility 123, build Report B and email it to just me and my client. – Date • Build Report C for facility 123 and email it to the entire group on the 15th of every month. .
  38. 38. Data Export to Visualization Programs
  39. 39. Thank You Jane Kennedy ARCADIS U.S., Inc. Phone: (504) 832-4174 Cell: (225) 205-8256 Email: jane.kennedy@arcadis- us.com