FME to the Rescue


Published on

DCP Midstream, a gas gathering and processing company, has over 60,000 miles of pipelines which are maintained in an enterprise GIS solution. The GIS database has over 158,000 linear pipeline segments in ten non-contiguous States. To import the pipelines into the "One Call, Call Before You Dig" software where only polygons are accepted, a query needed to run to capture only the company operated pipelines and to create a buffer. This process coulid not be handled by the GIS software itself, but FME came to the rescue. In this session, we will explore how FME was used to address this challenge.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

FME to the Rescue

  1. 1. Using FME to Overcome General GIS Software Limitations Alicia Foose, DCP Midstream
  2. 2. DCP Midstream Overview   DCP Midstream, LLC, a 50-50 joint venture between Spectra Energy and ConocoPhillips, is headquartered in Denver, Colorado.   The Company leads the midstream segment as one of the nation’s largest natural gas gatherers and processors in the United States.   DCP Midstream is the largest natural gas liquids (NGLs) producers in the nation.
  3. 3. DCP Midstream Overview   The Company owns or operates 58 plants, 10 fractionating facilities, and approximately 60,000 miles of gathering and transmission pipeline with connections to approximately 38,000 active receipt points.   Visit for more details.
  4. 4. What Does This Look Like
  5. 5. Close Up Look
  6. 6. GIS Environment   Oracle database   ESRI SDE   Pipeline Open Database Standard (PODS) 4.02 database model    The volume and complexity of data can create challenges for GIS analysis.
  7. 7. A Recent Project   DCP Midstream went through an evaluation process to find a software solution to manage the One Call (Call before you dig) process.   One of the solutions only accepted polygon features. Because all of the pipelines in the PODS database are polylines features a buffer needed to be created for the pilot.   To keep the comparisons similar we decided to buffer the pipeline by one foot.   Only the location of the pipelines were of interest.
  8. 8. Remember The 60,000 Miles
  9. 9. Volume Of Data   We were only interested in the pipelines we operated so a query was necessary.   The pipeline layer being used has 164,535 polylines in the database. SQL> select count(*) from PODS.REGULATORY_SEGMENT; COUNT(*) ---------- 164535   Laptop processing capacity along with memory limits can become an issue when buffering this volume of data.
  10. 10. Creating The Buffer In FME   The buffer was created in FME because   It is easy to set up   It can run in the background   It doesn’t seem to use as many resources   It tends to run faster on my environment   It has an aggregate feature   It can filter attributes
  11. 11. Creating The Buffer In FME   The goal was to:   Query for only DCP Midstream Operated pipelines.   Simplify the data by eliminating most of the columns.   I chose to keep Region because there are only 10 regions (Regions have a logical geographical area)   Buffer the pipelines by 1 foot.   Aggregate the data.   Export the Polygon feature to an ESRI shape file format.
  12. 12. Query For DCP Operated   The data was queried directly from the SDE connection in FME – this filters the data on the fly.   The 164,535 rows were reduced by 2,829 to total 161,706 records to buffer.
  13. 13. Transformers Used   The AttributeKeeper was used to reduce the number of columns from 51 down to 6 keeping only REGION_NAME from the SDE layer. Because only the location of a pipeline was required, the associated attributes were not needed.
  14. 14. Transformers Used   The Reprojector was used to project the data from NAD 83 to a projection with a unit of measure in feet.   US48-DUKE was chosen because the projection was created for the continental US and has relatively little overall distortion.
  15. 15. Transformers Used   The Bufferer was used to buffer by 1 foot.
  16. 16. Transformers Used   The Aggregator was used to aggregate the data using the REGION_NAME to group by.   Aggregating the data reduced the number of records from 161,706 to 10.
  17. 17. Transformers Used   The Reprojector was used to project the data back to NAD83.   Finally, the destination dataset was set to a shape file format. A visualizer was used so the output could be viewed right away.   A dissolver transformer was not used because the aggregate combined all of the polygons into Regions and the overlaps were not a concern for the end use.
  18. 18. FME Workspace
  19. 19. Final Results Total Features Written 10 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Translation was SUCCESSFUL with 0 warning(s) (10 feature(s)/5295013 coordinate(s) output) FME Session Duration: 7 minutes 9.2 seconds. (CPU: 141.9s user, 10.6s system) END - ProcessID: 1480, peak process memory usage: 208120 kB, current process memory usage: 53868 kB.
  20. 20. A Side By Side Comparison   A 1 foot buffer was run in Arc Info using the same query on the same layer.   Dissolve by field - REGION_NAME was selected because it is the closest option to the FME Aggregate .
  21. 21. A Side By Side Comparison Executing (Buffer_2): Buffer PODS.REGULATORY_SEGMENT Server 1_ft_Buffer.shp "1 Feet" FULL ROUND ALL # Start Time: Thu Mar 11 08:29:01 2010 Dissolving... Output feature 0 cannot be dissolved into other inputs because of memory limitations Output feature 1 cannot be dissolved into other inputs because of memory limitations … … Output feature 15 cannot be dissolved into other inputs because of memory limitations Executed (Buffer_2) successfully. End Time: Thu Mar 11 11:12:32 2010 (Elapsed Time: 2 hours 43 minutes 31 seconds)
  22. 22. A Side By Side Comparison   The FME translation ran in 7 minutes 9.2 seconds with no memory errors.   The Arc Info Buffer wizard ran in 2 hours 43 minutes and 31 seconds with memory limitations errors.   The results from either process were acceptable.
  23. 23. Annual Tax Project   Benjamin Franklin once said that “In this world nothing is certain but death and taxes”.   So lets talk about taxes, specifically property taxes. You might be asking yourself what on earth does FME have to do with property taxes. Well here is your answer-   Each year companies with tangible assets pay property taxes. Pipelines are not excluded.
  24. 24. The Challenge   Every State has unique taxing districts by which they collect and distribute property taxes.   Tax districts can change from year to year although most remain the same.   Population shifts and demographics are the most common cause of tax boundary changes.   DCP Midstream operates primarily in 17 States so tax boundary maintenance is a fairly large undertaking.
  25. 25. Tax Project   Each year the GIS department provides the Tax department with a report of how many feet of each pipeline is in what tax district by install year, diameter and so on.   The first step, for States with electronic data, is to download the current tax boundary files and update the SDE layer with the changes.   The SDE Layer has to be topologically clean.   Neighboring states do not tend to use the exact same state line. This creates gaps and overlaps which are ugly to clean up particularly along rivers.
  26. 26. 2008 Oklahoma Tax Districts
  27. 27. 2009 Oklahoma Tax Districts
  28. 28. Oklahoma Tax Districts   Who can tell me what changed?   Going once   Going twice   Going three time   How are you going to find out?   FME has a transformer named Matcher which detects both geometry and attribute changes from two files.
  29. 29. Lets See What Changed   The 2008 Oklahoma tax districts are added as one source.   The 2009 Oklahoma tax districts are added as another source.   Both are run through the Matcher as input.   The Not_Matched features are output to a visualizer so they can be looked at.   The Not_Matched features are output to a shape file to be used in ArcMap for updating the SDE layer.
  30. 30. FME Workspace
  31. 31. This Is What Changed
  32. 32. Why So Many Changes – River Correction
  33. 33. A Closer Look
  34. 34. Thank You!   Questions?   For more information:   Alicia Foose   DCP Midstream  