Your SlideShare is downloading. ×
MGCP4LCSS Workflow
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

MGCP4LCSS Workflow

330

Published on

Presented by Martin de Zuviria & Amanda Chiprout, National Defense, Government of Canada …

Presented by Martin de Zuviria & Amanda Chiprout, National Defense, Government of Canada
See more FME World Tour 2014 presentations at www.safe.com/recap2014

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
330
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • The MGCP4LCSS workflow performs an extraction of selected features contained in the MGCP vector data, and to transform them to meet the schema and visualization requirements for LCSS Battleview. The final output was a FGDB using representations as symbology to be consumed in a WMS/WFS.This data transformation process: - initial geometry and schema validation, - feature merging or segregation - assignment of descriptive names to replace numeric codes. - load the transformed data into an empty file geodatabase (FGDB) containing symbology rules required to represent data using the cartographic style defined by Topographic Line Maps (TLM).The workflow created by Martin provides a step-by-step description on how a user may proceed from beginning to end. The workflow has been proven to work in ArcGIS Data Interoperability.
  • There are different ways to filter data with FME: 1) readers; 2) transformers such as ‘TestFilters’ and ‘GeometryFilters’; 3) enabling/disabling components of the FME workspace (e.g. by enabling only all objects within the Road_L bookmark)QA/QC may be done by re-directing results to Inspector and using another reference layer(s), analyzing log files. In the next slide we show an example
  • Note: ‘Data Quality’ is regarded here as the degree of excellence of data to satisfy a given objective. Any tile of MGCP released has to pass QAs, but MGCP data has to meet new requirements of a transformation to a FGDB format/schema and Web Mapping Services and therefore has to be re-tested for these new requirements, regardless of previous QAs performed before generating the input SHPs.OGC standards for different types of Web Mapping Services are outlined at http://www.opengeospatial.org/standards/wfsThese standards may be different for different types of WMS/WFSs but there are general validations that may be considered the most relevant for most of them, and OGC standards are widely accepted by the international community.One of these standards related to geometry validations is to test for ‘self-intersections’ of linear features (relevant since a point can not be a vertex and a node simultaneously in the output FGDB). A tester could be tunned to only detect or to detect and repair, and in this case it would split lines and copy the attribute values to each resulting line. It is in general a practice I would recommend to define the types of repairs accepted for features that have to undergo future transformations afterwards. Another valid example in this case of roads relates to attributes: there were a couple of road segments that were not mapped as any of the valid classes defining when applying the rules provided to define feature classes for ‘Roads’ and this segments were output as “Road types not properly defined”. If no action would have been implemented at this point in the ETL workflow, these two segments would have been missing in the destination FGDB
  • In the legend of the output map document, we may see that ‘Extraction Mine Areas’ and ‘Quarry Areas’ have been merged into a single feature class. At the bottom we may see that descriptive attributes have replaced several coded valuesFeature Merging along with modifying schema to meet the target architecture (merging ‘Extraction Mine Areas’ and ‘Quarry Areas’, managing schema and coding symbology – Steps 3,5,6)
  • along with modifying schema to meet the target architecture (merging ‘Extraction Mine Areas’ and ‘Quarry Areas’, managing schema and use of descriptive names and coding symbology using the ‘RepID’ field – Steps 3,5,6)
  • Custom transformers were used to replace a sequence of transformers used in the workflow. Some of these transformers contained long LUTs used to convert numeric values to descriptive names. Since these LUTs may be regularly updated by having new source values like in the example shown on the slide, the ability to update them only once in the workflow saves time and reduces the risk of errors.
  • FME Output FGDB is loaded into an empty FGDB with identical schema plus the desired Representations, and symbology IDs are linked to Representation Rules (values from ‘RepID’ are copied to the ‘RuleID’ field, tied to representation rules)
  • Transcript

    • 1. MGCP4LCSS Workflow Engineering Section Mapping & Charting Establishment 1 Martin de Zuviria Amanda Chiprout
    • 2. BattleView Overview
    • 3. 5 MGCP4LCSS OVERVIEW The main GOAL of the workflow (MGCP4LCSS) is to provide the LCSS user with the ability to view and interact with Symbolized MGCP vector data through multiple Web Services. The Geospatial Data Manager will need three items to execute this process: - MGCP vector data in Shapefile format - FME Desktop - ArcGIS software applications
    • 4. MGCP4LCSS WORKFLOW  The MGCP4LCSS workflow performs: – an extraction of selected features contained in the MGCP vector data – transform them to meet the schema and – visualization requirements for LCSS Battleview. – The final output was a FGDB using representations as symbology to be consumed in a WMS/WFS.  The workflow created by Martin provides a step-by- step description on how a user may proceed from beginning to end.  The workflow has been proven to work in ArcGIS Data Interoperability.
    • 5. Web Optimization  MGCP data is rich with content  MGCP not optimized for Web Services  Schema Optimized for Web: – Cut feature count in half – ‘Human Readable’ – Scaled to GRLS (Global/Regional/Local/Specialized) – Symbolized using Topographic Line Maps
    • 6. 8 Blue is data conditioning team Green is the Geo Field Teams MCE MGCP4LCSS Production Process Input: MGCP Shapefiles IGW Filter Shapefiles & categorization Pre-Conversion Check – QC/QA Aggregate Features Segregate Features Feature Description Symbology Coding Load & Calculate FGDB Battleview schema with representations FME 2013 FME 2013 FME 2013 FME 2013 FME 2013 FME 2013 Arc GIS 10.1 IMO/Chief Geo Receives FGDB DMAN Team Ingest/ load on Geo Server WFS Services built and tested Live on LCSS in operational theatre
    • 7. FME Workspace An FME workspace contains source and destination types of data and attributes, as well as transformers that manipulate data. Bookmarks have been added to easily find and re-focus the main display on any destination feature class (e.g. Helipad_P) Custom transformers, represented in green color, have also been added to allow an easy update of the settings of specific transformers that are of common use within the same workspace (e.g. ‘Source Name’) 10
    • 8. Step 1 : Feature Filter 11 There are different ways to filter data with FME: 1) Readers 2) Transformers such as ‘TestFilters’ and ‘GeometryFilters’; 3) Enabling/disabling components of the FME workspace (e.g. by enabling only all objects within the Road_L bookmark)
    • 9. Step 2: QA/QC Workspace 12 QA/QC results are: - re-directed results to an Inspector - using other reference layer(s), - analyzing log files. Note: ‘Data Quality’ is regarded here as the degree of excellence of data to satisfy a given objective. Any tile of MGCP released has to pass extensive QA. MGCP data has to meet new requirements in its transformation to a FGDB format/schema and Web Mapping Services (OGC Standards) and therefore has to be re-tested for these new requirements, regardless of previous QA performed.
    • 10. Steps 3,5,6: Feature Merge/Aggregate Output 13 Feature Merging and aggregation to meet the target schema architecture: - merging (ie ‘Extraction Mine Areas’ and ‘Quarry Areas’) - managing schema - coding symbology Example Below: In the legend of the output map document, the ‘Extraction Mine Areas’ and ‘Quarry Areas’ have been merged into a single feature class. The bottom table shows the descriptive attributes replaced several coded values
    • 11. 14 Steps 3,5,6: Feature Merge/Aggregate Output Example in FME Example: - Merge ‘Extraction Mine Areas’ and ‘Quarry Areas’, - manage schema and - Descriptive names and coding symbology using the ‘RepID’ field
    • 12. Usefulness of Custom Transformers 15 Custom transformers were used to replace a sequence of transformers used in the workflow. Some of these transformers contained long LUTs used to convert numeric values to descriptive names. Since these LUTs may be regularly updated by having new source values like in the example shown on the slide, the ability to update them only once in the workflow saves time and reduces the risk of errors.
    • 13. 16 Steps 4,5,6: Feature Segregation and Descriptive Names: Building points are segregated into a new feature class and given ‘Human Readable Names’ i.e. ‘Transportation_Station’ and ‘Vertical_Obstruction’
    • 14. 17 Steps 4,5,6: Feature Segregation and Descriptive Names: Building points are segregated into a new feature class and given ‘Human Readable Names’ i.e. ‘Transportation_Station’ and ‘Vertical_Obstruction’
    • 15. Step 7: Feature Loading to the Destination Empty FGDB and Calculation of Symbology Representation Values 18 FME Output data is loaded into an empty FGDB with identical schema plus the desired Representations, and symbology IDs are linked to Representation Rules (values from ‘RepID’ are copied to the ‘RuleID’ field, tied to representation rules) Example: ArcGIS ETL Detail (partial – created using ArcGIS Model Builder)
    • 16. MGCP4LCSS V3 Workflow: Information Structure Step 1: Create Parent Folder Structure Step 2: Create Sub Folder where MGCP input SHPs must be copied Step 3: Subfolder Output FGDBs: - The contents of subfolders with empty FGDBs must be copied here Step 4: Run Workbenches and BatchRunner ETL for a ‘single click’ processing of steps 1 to 6 Step 6: Subfolder ‘ArcGIS Output Map’ with an ArcGIS Map for visualization of output Step 5: ‘ArcGIS Toolboxes’ containing ArcGIS ETL Tools for processing of step 7
    • 17. 20 MGCP4LCSS Workflow Pre-conditions The designated Geospatial Data Manager has copied the ‘MGCP4LCSS Package’ to his/her local C:/ drive and has FME Desktop (2013 release or later) and ArcGIS (10.1 release or later) installed on his/her computer Trigger The user wants to covert MGCP tiles provided as shapefiles to the FGDB schema and representations defined by DND MCE Engineering, for subsequent use in any LCSS application Main Flow 1. Copy the subfolder /MGCP of the ‘MGCP4LCSS’ package on the local C:/ drive 2. Copy the MGCP tiles to the C:/MGCP/INPUT MGCP TILES folder 3. Open FME Desktop and run each of the individual eight workspaces contained in the subfolder ‘FME Workbenches’ 4. Open ArcGIS – ArcCatalog and runs the six tools contained in the Toolbox C:/MGCP/ArcGIS Toolboxes/Battleview.tbx, starting with the three ‘Append’ tools 5. Open ArcMAP MXD BVoutputFGDB.mxd to visualize the results from the output FGDB
    • 18. MGCP4LCSS SOP 21

    ×