Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automated Bulk Conversion of 250,000+ CAD and Corresponding Raster Files to PDF

946 views

Published on

This presentation will describe the process and results of a contracting project that utilized both FME Desktop and FME Server to combine more than 250,000 DGN files and associated rasters and referenced border files into PDF files that can be easily viewed via mobile devices. The purpose of the project was to provide the client with an automated process that enables mobile access to their Substation, Transmission, and Business Protection designs.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Automated Bulk Conversion of 250,000+ CAD and Corresponding Raster Files to PDF

  1. 1. Automated Bulk Conversion of 250,000+ CAD and Corresponding Raster Files to PDF Courtney Maxson
  2. 2. Courtney Maxson • Certified FME Professional • Certified FME Trainer • M.S. GIS, Johns Hopkins • SBS Employee – 2.5 years Spatial Business Systems • Spatial integration firm based in Lakewood, CO • FME Gold-level reseller • 5 certified FME professionals and 2 trainers
  3. 3. PROBLEM? • Utility with 250,000 + MicroStation DGN’s • DGN’s only viewable via Bentley Viewer • Not compatible with mobile devices • Access needed in field • Manual creation process What’s the
  4. 4. Goal? • Convert DGN’s and associated TIFF’s and Borders to PDF files • Process multiple DGN’s simultaneously • Produce 1 PDF per DGN with few exceptions • Customize output based on DGN type • Automate PDF conversion process What’s the
  5. 5. PDF Production Process Diagram
  6. 6. SOLUTION FME Empty Temp Directory Copy Files Merge X-ref’s Generate PDF’s 01 02 03 04
  7. 7. PDF Production Process Diagram
  8. 8. The Workspaces
  9. 9. Runner Workspace Workspace Parameters: § TEMP_DIR § SOURCE_DATASET_CSV § SOURCE_DATASET_CTL_DGN § TYPES_TO_COPY_MERGE § DELTAS_ONLY
  10. 10. Source Paths CSV
  11. 11. Step 1: Copy Files to Temp Directory Startup Python Script § Empties Temp directory § Reads DGN directories via CSV and Directories and File Pathnames Readers § Copies DGN’s to Temp directory § Copies TIFF’s to Temp directory
  12. 12. Step 2: Merge Temp DGN’s C# Script Functions: § Merge DGN’s and borders § Extract values for TIFF alignment • Temp path • Lower Level X,Y • Dimension Y • Pixel size X,Y § Create RasterTable.csv
  13. 13. Step 3: Create PDF Files
  14. 14. PDF Conversion Workspace Workspace Parameters: § SOURCE_DATASET_DGNV8 § SOURCE_DATASET_TIFF § SOURCE_DATASET_CSV § Converts DGN to single PDF with associated borders and/or TIFF’s § Can process multiple file sets simultaneously § Generates multiple PDF’s from single source DGN, when required
  15. 15. PDF Conversion: Read DGN Elements
  16. 16. PDF Conversion: Manipulate DGN Text Issues / Workarounds § Dimension Text Bug • Manual conversion § Degree (°) Not available in font • Convert to rich text § XML escape characters • Converted to escape string • & replaced with & § Ω and Φ not supported by PDF Writer • Replaced with text OHM and PH
  17. 17. PDF Conversion: Manipulate Non-Text § Remove multi-text nodes § Resolve Line Style domains
  18. 18. PDF Conversion: Manipulate TIFF Issues / Workarounds § TIFF alignment • Use RasterTable.csv created by C# script § TIFF’s processing without DGN § Multiple DGN’s with same name • Add directory path to RasterTable.csv • Create fanout_attr
  19. 19. PDF Conversion: Style and Write to PDF For each non-Z Subeng DGN: § Queries Subeng DB to determine # of output PDF’s § Removes unneeded levels for each plan type § Creates 1 PDF per plan
  20. 20. PDFFormatWriter Transformer Transformer Parameters: § FME_PROCESS_GROUP_BY § TARGET_DATASET § Determines extent of all elements per fanout_attr § Calculates extent orientation § Formats PDF based on orientation § Writes DGN and TIFF elements to different layers
  21. 21. PDF Results System Protection
  22. 22. PDF Results Substation
  23. 23. Substation One Line
  24. 24. PDF Results Transmission
  25. 25. Initial Run So many files, so little time
  26. 26. FME Server Solution • Distributed environment • 3 Production Servers • 10 Engines, 1 Core • Job Routing • Production Server: o Dual-core CPU o 8 GB RAM o 2 Permanent Engines • Temporary Production Server 1: o Quad-core CPU o 16 GB RAM o 4 Temporary Engines • Temporary Production Server 2: o Quad-core CPU o 16 GB RAM o 4 Temporary Engines fmeServerConfig.txt
  27. 27. Performance Estimated Ctldgn Subeng Trans TOTAL Total DGN 122745 28561 551 151857 Avg Time/Job (Sec) 12 26 15 Total Est Hrs - 1 Engine 409.15 206.27 2.29 617.71 Total Est Hrs - 10 Engines 61.771
  28. 28. Automation Long-Term Solution
  29. 29. FME Server • 1 Production Server • 2 Engines • Scheduler - Runs every hour • DELTAS_ONLY = Yes Long-Term Solution
  30. 30. Deltas Only Selection For each non-Z Subeng DGN: § Queries Subeng DB to determine # of output PDF’s § Removes unneeded levels for each plan type § Creates 1 PDF per plan • Only processes new/modified DGN’s • Compares modified dates of DGN and PDF
  31. 31. Final Steps • On Production Servers: • Switch to Divided Installation? • Finish initial run • Test DELTAS_ONLY Process • Setup Schedule
  32. 32. FME is the Solution! With a little TLC J
  33. 33. THANK YOU! www.spatialbiz.com courtney.maxson@spatialbiz.com

×