Around the World with FME
Multilevel
Tile Cache
Generation
Engine
LA RIOJA, SPAIN
Ana García de Vicuña
Pablo Martínez
Gobierno de La Rioja
MBTiles format
Store pre-cache tiles in a sqlite database to speed web mapping
Implemented in the workbench
Mapnik
- Toolkit for making high quality raster maps
- Complex styles and simbology
- FME transformer since FME 2014 (MapnikRasterizer)
Multilevel tile cache generation engine
- Raster data
- Vector data
- Text labels
- Parameters (zoom, bbox,…)
MapnikRasterizer =
f(zoom)
Input Apply symbology
WebMaptiler
(GoogleMaps compatible)
Tiled data
Multilevel tile cache generation engine
Save unique tiles
Optimizing MBTiles MBTiles database
Mbtiles
f(zoom)
Output
- metadata
- images
- maps
tiles
Water
60%
Land
38%
Mix
2%
Tiles TMS
Tables View
http://bit.ly/iderioja
Multilevel tile cache generation engine
Ana García de Vicuña
Pablo Martínez
Thank you very much!!!
See you in Barcelona…
View the full presentation at
http://goo.gl/0dqjR4
(en español)
3D DGN to
Multipatch
for City
Engine
UNITED KINGDOM
Konrad Poplawski
Thames Tideway Tunnel
Needed:
 a reliable, repeatable
method to transfer 3D
model data between
Bentley MicroStation
and Esri City Engine.
Challenges:
 no direct format
support
 georeferencing
 attribution
Basics
 Collada chosen as an interim
format
 DGN is georeferenced
 Python script reads file pairs
based on name
 Batch deploy processes full
sets of models
Maintaining Georeferencing
 BoundingBoxAccumulator
 CenterPointReplacer
Attribute Handling
 Create attribute lists from original CAD file
 Generate keys for downstream re-attribution
 Clean up attributes
Re-attribution
 asdf
The Result: 3D Web View
 Esri Multipatch
Feature Classes,
positioned and with
attributes
 Detailed planned
construction available
to all stakeholders in
a browser
Data
Migration at
Irish Water
IRELAND
Patrick Daly
Irish Water
The Project
 Irish Water formed in
2013 to bring together
water and wastewater
services of 34 Local
Authorities
 Networked data including
nearly 80,000 km of pipe
and hundreds of
thousands of point
features
Challenges
 Source: Local
authority data in CIS
and MapInfo datasets
 Not in consistent
formats
 30+ variants in format
and schema
Water Below Ground
Selected Feature Class Feature Count Length /km
Air Valve 34,479
Fittings 493,798
Flow Control Valves 2,932
Hydrants 148,451
Network Meters 28,221
System Valves 229,623
Laterals (Service/Comms) 307,003 6,293
Water Mains (of which 6,055Km
Private) 713,390 61,183
Solving with FME
 Destination: complex
ArcGIS Water Utility
Data Model
 30+ sets of
workspaces for
normalization and
migration
 15 hours processing
time
Destination: IW Enterprise GIS
Irish Water
We completed this project in 9 weeks using FME, migrating
almost 80,000 km of Water & Waste Water Network data. This
scale of project with this timeline would not be achievable
without FME.
Patrick Daly, Asset Register & Data Aggregation Specialist
Irish Water
Irish Water GIS Migration Team
Kelly Brady – GIS Analyst North West
Sean Minogue – GIS Analyst East/Midlands
Sandra Nestor - GIS Analyst East/Midlands
Rónán O’Shea – GIS Analyst South
Mark Healy – Reporting & Information Analyst
BIM to GIS
at Mount
Vernon
VIRGINIA, USA
Patrick Gahagan, Esri
Quinn Evans Architects
Mount Vernon Ladies’ Association
Mount Vernon
 George Washington’s home, constructed in the late 1800s
 Mount Vernon Ladies’ Association tasked with restoration, interpretation,
and preservation of grounds and structures
 Mansion laser scanned to create architectural-quality HBIM in Revit by
Quinn Evans Architects
BIM to GIS via Data Interop
 Revit: Add coordinates and
rotation from project north to
true north
 Export to Revit Archive(.rvz)
with FME Revit Exporter,
attributes to spreadsheet
 Import to ArcGIS with Data
Interoperability Extension
(FME)
 Reconnect attribution
A Blended World
Viewshed Analysis
 asdf
Stakeholder Access to Information
 Browser delivery provides
data to everyone, even
fire suppressant system
designer
 Historical data identifies
plaster from 1950 vs.
1787
 Framing details assist with
optimal routing
A navigable, queriable world provides minute detail and the big picture to all
stakeholders – BIM/GIS pros or not.
OS 3D Data
Generation:
Using FME to Investigate
the Creation of Point
Clouds from Imagery
UNITED KINGDOM
Nikki Goodwyn
Photogrammetric Surveyor
Ordnance Survey
THE CHALLENGE
 SYNCHRONISING TWO OUTPUTS: DIGITAL SURFACE MODEL
(DSM) AND DIGITAL TERRAIN MODEL (DTM)
DSM DTM
METHODS
Point Cloud- No Colour ValuesRGB colouring brings points back to reality
SOLUTION
FME WORKBENCH
INPUT 1:
ORTHO
GEOTIFF
FME
EXTRACTS
Raster
Properties
Including
RGB
Properties
Combined into
RGB & Real64
FME
CONVERTS
DEM (XYZ)
INT32 TO
REAL64
INPUT 2:
DSM
GEOTIFF
Scaled and
Output as
LAZ Point
Cloud
SOLUTION
Input =1km Ortho Tile as
Geotiff
Input= 1km DSM of same
Tile as Geotiff
RasterBandInterpretation
Coercer
RasterPropertiesExtractor
RasterExpressionEvaluator
RESULTS
COLOURISED IMAGE-BASED POINT CLOUD OUTPUT
Nikki Goodwyn
NGoodwyn@os.uk
02380 055972
THANK YOU
UAV-Based
LiDAR Data
Collection &
Analysis
FINLAND
Ville Koivuranta
Sharper Shape Ltd.
Next Eagle® by Sharper Shape
 Our service fully automatically detects each vegetation issue that
threatens the transmission network. We prioritize and visualize
the vegetation observations to create ready management plans
and work orders that customer could send to his subcontractors.
 We identified 3-5 times more issues in the areas that our solution
inspected than with traditional methods.
 This enabled need based vegetation management.
UAV Solution
 Sharper Shape UAV’s are capable Beyond Visual Line Of Sight
flights with up to 1 hour flight time and up to 8 kg payload.
 Sharper Shapes UAV’s are equipped with high performance LiDAR,
High resolution cameras, onboard computer and storage unit and
high precision IMU (Inertial Measurement Unit)
FME in Route Planning
 Economically optimized flight routes are created using FME.
 The elevation information is sourced from existing point
cloud or from national DTM.
 3D shape polylines are uploaded to autopilot and the UAV
follows the pre programmed flight line.
Customer Data Integration
 FME is perfect to fulfill customer specific small needs.
 For example one customer wanted to be able to create cross-section
images from network and it was done using FME
 Also all data conversions from customer NIS system to our systems are
done with FME.
 FME is used to create PDF-map of the vegetation issues that can be used
field personnel to do the vegetation clearances.
Prioritization Problem
 Distribution companies can have hundreds of thousands of
vegetation issues.
 Vegetation management need to be targeted to where it is most
needed.
 Economically optimized vegetation clearance plans created with
FME can reduce expected losses caused by power interruptions up
to 3-5 times more than legacy methods.
Vegetation Management Prioritization
Handle the areas in the
priority order. Proceed with
planning.
Conclusions
 Combination of these new
technologies allows us to take
full control of our asset
management.
 Our goal is to create full turn
key solution that is both cheaper
and more effective than any
legacy method.
Building a
Spatial Decision
Support System
for Natural Gas
Pipeline Risk
OKLAHOMA, USA
Matt Landry
Frank Yeboah
Williams Companies
Gas
Gathering System
R_GGS_FACTS.fmw
[R_GGS_FACTS]
SEGMENT_ID
R_SEGID_FACTS.fmw
[R_SEGID_FACTS]
GISID_ID
R_GISID_FACTS.fmw
[R_GISID_FACTS]
SUBSEG_ID
R_SUBSEGID_FACTS.fmw
[R_SUBSEGID_FACTS]
R_SUBSEGID_FACTS.fmw
Image Source:
http://www.utc.wa.gov/publicSafety/
Documents/Williams%20Presentation
%20-%20ILI%20Technologies.pdf
Leveraging
FME Cloud:
Near Real Time
Global Web Map
Tile Generation for
Meteorology
CANADA
Pelmorex
The Project
Produce web map tiles from a worldwide forecast
meteorological model called ECMWF. Layers include:
 Precipitation
 Sea Surface Temperatures
 Ground Temperatures
 Wind speed and direction
Challenges
 880,000 tiles need regenerating every 12 hours.
 The maps are time sensitive so the data needs
processing as quickly as possible.
 Each run needs around 80 hours of compute time.
 Their on-premises 10 engine FME Server did not
have powerful enough hardware.
Solution
 Leverage FME Cloud elastic processing workflows.
 Provision FME capacity dynamically every 12 hours.
 Leverage AWS services such as S3, SQS and AWS
Lambda.
Solution – Dynamically Provision Capacity
Solution – Processing Data using SQS
Cost Analysis
 Costs $80 per run or $58,000 annually.
 On FME Server to replicate performance they
would need 40 engines (~ $300,000) and a lot of
hardware.
Oracle
Spatial to
SAP HANA at
Alliander
NETHERLANDS
Stefan Koster
BI&A Architect
Oracle Spatial to SAP HANA at
Alliander
The Challenge
 Alliander wants repeatable workflows to move
data from Oracle Spatial/Esri Geo Data
warehouse to SAP HANA for advanced SAP/GEO
BI analysis
 Departments aligning, GIS has FME already
 FME preferred, complex spatial handling
 Will FME do the job?
JDBC Technology Preview
 Java Database
Connectivity tech
preview in FME
2015
 Configured for
SAP HANA
 Transformers
massage
geometry into
expected format
The Results
How did FME do?
 Complete ETL process 2.5 to 3 times faster
than the tested alternative
 Advanced spatial handling of FME enables
automation
 ETL tasks across departments on a common
platform
Geostor:
State GIS
Clearinghouse
Cloud Migration
ARKANSAS, USA
Seth LeMaster
Tony Davis
Arkansas GIS OfficePhoto by Chriseast18/ CC BY
Overview
Arkansas GIS Clearinghouse
 GIS open data portal for the state of Arkansas
 1222 monthly downloads (2,358 items downloaded)
 Users
 108 registered
 2,387 non-registered
Overview
The Project - GeoStor
Improve the usability and migrate to the cloud.
 300 vector datasets migrated to PostGIS and AWS
S3
 3TB of raster data
 4TB of historical raster data to AWS Glacier
Architecture
The Workspace
Benefits of Cloud
 Stability – Fault tolerant data storage and Safe
monitor and support FME Cloud architecture.
 Security – Leverage AWS compliance and FME Cloud
security policies.
 Simplicity – Focus on problem not the administration
 Price – 3 times cheaper than on-premises
Option 1 – On Site Upgrade
Monthly Yearly Up-Front
DIS Server Space $3,200.00
FME Dekstop $6,000.00
FME Server $12,000.00
Dell Hardware $100,000.00
SQL Server
Windows Server Std (Software Assurance)[5] $595.00
Windows Sevrer Enterprise (Software Assurance) [1] $385.00
Microsoft SQL Server (Software Assurance) $270.00
Microsoft SQL Server Standard Edition (Licenses) $294.00
Microsoft Server Std Edition (License) [6] $1,662.00
Microsoft Windows Server Enterprise Ed (Licences)
[5] $4,490.00
Symantec $680.00
Tape Backups (No server) $2,000.00
Total Recurring Montly Payment $3,200.00
Total Recurring Yearly Payment $26,376.00
Total One Time Cost (Every 3 yrs) $102,000.00
Total Three Year Cost $296,328.00
True Monthly Cost (/36 months) $8,231.33
Intangiable Costs
Hardware Maintenance Time
DIS Process
Non-Scaleable
Option 2 – Cloud
Monthly Yearly Up-Front
FME Dekstop $6,000.00
FME Cloud $14,400.00
EC2 Instance (m3.large) $395.81
AWS Storage (EBS 780GB) $77.11
AWS Storage (S3 2TB) $61.30
AWS Storage (Glacier 4.2 TB) $42.41
Total Recurring Montly Payment $576.63
Total Recurring Yearly Payment $20,400.00
Total Three Year Cost $81,958.68
True Monthly Cost (/36 months) $2,276.63
Our rack space costs (real estate on our data center floor) $3,800 per month. Add to that the hardware costs,
etc and you start to see why moving to the cloud was a no brainer for us. Anthony Davis, State Arkansas
Thank You!
 Questions?
 blog.safe.com

FME Around the World

  • 1.
  • 2.
    Multilevel Tile Cache Generation Engine LA RIOJA,SPAIN Ana García de Vicuña Pablo Martínez Gobierno de La Rioja
  • 3.
    MBTiles format Store pre-cachetiles in a sqlite database to speed web mapping Implemented in the workbench
  • 4.
    Mapnik - Toolkit formaking high quality raster maps - Complex styles and simbology - FME transformer since FME 2014 (MapnikRasterizer)
  • 5.
    Multilevel tile cachegeneration engine - Raster data - Vector data - Text labels - Parameters (zoom, bbox,…) MapnikRasterizer = f(zoom) Input Apply symbology WebMaptiler (GoogleMaps compatible) Tiled data
  • 6.
    Multilevel tile cachegeneration engine Save unique tiles Optimizing MBTiles MBTiles database Mbtiles f(zoom) Output - metadata - images - maps tiles Water 60% Land 38% Mix 2% Tiles TMS Tables View
  • 7.
  • 8.
    Ana García deVicuña Pablo Martínez Thank you very much!!! See you in Barcelona… View the full presentation at http://goo.gl/0dqjR4 (en español)
  • 9.
    3D DGN to Multipatch forCity Engine UNITED KINGDOM Konrad Poplawski
  • 10.
    Thames Tideway Tunnel Needed: a reliable, repeatable method to transfer 3D model data between Bentley MicroStation and Esri City Engine. Challenges:  no direct format support  georeferencing  attribution
  • 11.
    Basics  Collada chosenas an interim format  DGN is georeferenced  Python script reads file pairs based on name  Batch deploy processes full sets of models
  • 12.
  • 13.
    Attribute Handling  Createattribute lists from original CAD file  Generate keys for downstream re-attribution  Clean up attributes
  • 14.
  • 15.
    The Result: 3DWeb View  Esri Multipatch Feature Classes, positioned and with attributes  Detailed planned construction available to all stakeholders in a browser
  • 16.
  • 17.
    The Project  IrishWater formed in 2013 to bring together water and wastewater services of 34 Local Authorities  Networked data including nearly 80,000 km of pipe and hundreds of thousands of point features
  • 18.
    Challenges  Source: Local authoritydata in CIS and MapInfo datasets  Not in consistent formats  30+ variants in format and schema Water Below Ground Selected Feature Class Feature Count Length /km Air Valve 34,479 Fittings 493,798 Flow Control Valves 2,932 Hydrants 148,451 Network Meters 28,221 System Valves 229,623 Laterals (Service/Comms) 307,003 6,293 Water Mains (of which 6,055Km Private) 713,390 61,183
  • 19.
    Solving with FME Destination: complex ArcGIS Water Utility Data Model  30+ sets of workspaces for normalization and migration  15 hours processing time
  • 20.
  • 21.
    Irish Water We completedthis project in 9 weeks using FME, migrating almost 80,000 km of Water & Waste Water Network data. This scale of project with this timeline would not be achievable without FME. Patrick Daly, Asset Register & Data Aggregation Specialist Irish Water Irish Water GIS Migration Team Kelly Brady – GIS Analyst North West Sean Minogue – GIS Analyst East/Midlands Sandra Nestor - GIS Analyst East/Midlands Rónán O’Shea – GIS Analyst South Mark Healy – Reporting & Information Analyst
  • 22.
    BIM to GIS atMount Vernon VIRGINIA, USA Patrick Gahagan, Esri Quinn Evans Architects Mount Vernon Ladies’ Association
  • 23.
    Mount Vernon  GeorgeWashington’s home, constructed in the late 1800s  Mount Vernon Ladies’ Association tasked with restoration, interpretation, and preservation of grounds and structures  Mansion laser scanned to create architectural-quality HBIM in Revit by Quinn Evans Architects
  • 24.
    BIM to GISvia Data Interop  Revit: Add coordinates and rotation from project north to true north  Export to Revit Archive(.rvz) with FME Revit Exporter, attributes to spreadsheet  Import to ArcGIS with Data Interoperability Extension (FME)  Reconnect attribution
  • 25.
  • 26.
  • 27.
    Stakeholder Access toInformation  Browser delivery provides data to everyone, even fire suppressant system designer  Historical data identifies plaster from 1950 vs. 1787  Framing details assist with optimal routing
  • 28.
    A navigable, queriableworld provides minute detail and the big picture to all stakeholders – BIM/GIS pros or not.
  • 29.
    OS 3D Data Generation: UsingFME to Investigate the Creation of Point Clouds from Imagery UNITED KINGDOM Nikki Goodwyn Photogrammetric Surveyor Ordnance Survey
  • 30.
    THE CHALLENGE  SYNCHRONISINGTWO OUTPUTS: DIGITAL SURFACE MODEL (DSM) AND DIGITAL TERRAIN MODEL (DTM) DSM DTM
  • 31.
    METHODS Point Cloud- NoColour ValuesRGB colouring brings points back to reality
  • 32.
  • 33.
    INPUT 1: ORTHO GEOTIFF FME EXTRACTS Raster Properties Including RGB Properties Combined into RGB& Real64 FME CONVERTS DEM (XYZ) INT32 TO REAL64 INPUT 2: DSM GEOTIFF Scaled and Output as LAZ Point Cloud SOLUTION Input =1km Ortho Tile as Geotiff Input= 1km DSM of same Tile as Geotiff RasterBandInterpretation Coercer RasterPropertiesExtractor RasterExpressionEvaluator
  • 34.
  • 35.
  • 36.
  • 37.
    Next Eagle® bySharper Shape  Our service fully automatically detects each vegetation issue that threatens the transmission network. We prioritize and visualize the vegetation observations to create ready management plans and work orders that customer could send to his subcontractors.  We identified 3-5 times more issues in the areas that our solution inspected than with traditional methods.  This enabled need based vegetation management.
  • 38.
    UAV Solution  SharperShape UAV’s are capable Beyond Visual Line Of Sight flights with up to 1 hour flight time and up to 8 kg payload.  Sharper Shapes UAV’s are equipped with high performance LiDAR, High resolution cameras, onboard computer and storage unit and high precision IMU (Inertial Measurement Unit)
  • 39.
    FME in RoutePlanning  Economically optimized flight routes are created using FME.  The elevation information is sourced from existing point cloud or from national DTM.  3D shape polylines are uploaded to autopilot and the UAV follows the pre programmed flight line.
  • 40.
    Customer Data Integration FME is perfect to fulfill customer specific small needs.  For example one customer wanted to be able to create cross-section images from network and it was done using FME  Also all data conversions from customer NIS system to our systems are done with FME.  FME is used to create PDF-map of the vegetation issues that can be used field personnel to do the vegetation clearances.
  • 41.
    Prioritization Problem  Distributioncompanies can have hundreds of thousands of vegetation issues.  Vegetation management need to be targeted to where it is most needed.  Economically optimized vegetation clearance plans created with FME can reduce expected losses caused by power interruptions up to 3-5 times more than legacy methods.
  • 42.
    Vegetation Management Prioritization Handlethe areas in the priority order. Proceed with planning.
  • 43.
    Conclusions  Combination ofthese new technologies allows us to take full control of our asset management.  Our goal is to create full turn key solution that is both cheaper and more effective than any legacy method.
  • 44.
    Building a Spatial Decision SupportSystem for Natural Gas Pipeline Risk OKLAHOMA, USA Matt Landry Frank Yeboah Williams Companies
  • 48.
  • 49.
  • 51.
  • 53.
    Leveraging FME Cloud: Near RealTime Global Web Map Tile Generation for Meteorology CANADA Pelmorex
  • 54.
    The Project Produce webmap tiles from a worldwide forecast meteorological model called ECMWF. Layers include:  Precipitation  Sea Surface Temperatures  Ground Temperatures  Wind speed and direction
  • 55.
    Challenges  880,000 tilesneed regenerating every 12 hours.  The maps are time sensitive so the data needs processing as quickly as possible.  Each run needs around 80 hours of compute time.  Their on-premises 10 engine FME Server did not have powerful enough hardware.
  • 56.
    Solution  Leverage FMECloud elastic processing workflows.  Provision FME capacity dynamically every 12 hours.  Leverage AWS services such as S3, SQS and AWS Lambda.
  • 57.
    Solution – DynamicallyProvision Capacity
  • 58.
  • 59.
    Cost Analysis  Costs$80 per run or $58,000 annually.  On FME Server to replicate performance they would need 40 engines (~ $300,000) and a lot of hardware.
  • 60.
    Oracle Spatial to SAP HANAat Alliander NETHERLANDS Stefan Koster BI&A Architect
  • 61.
    Oracle Spatial toSAP HANA at Alliander
  • 62.
    The Challenge  Allianderwants repeatable workflows to move data from Oracle Spatial/Esri Geo Data warehouse to SAP HANA for advanced SAP/GEO BI analysis  Departments aligning, GIS has FME already  FME preferred, complex spatial handling  Will FME do the job?
  • 63.
    JDBC Technology Preview Java Database Connectivity tech preview in FME 2015  Configured for SAP HANA  Transformers massage geometry into expected format
  • 64.
  • 65.
    How did FMEdo?  Complete ETL process 2.5 to 3 times faster than the tested alternative  Advanced spatial handling of FME enables automation  ETL tasks across departments on a common platform
  • 66.
    Geostor: State GIS Clearinghouse Cloud Migration ARKANSAS,USA Seth LeMaster Tony Davis Arkansas GIS OfficePhoto by Chriseast18/ CC BY
  • 67.
    Overview Arkansas GIS Clearinghouse GIS open data portal for the state of Arkansas  1222 monthly downloads (2,358 items downloaded)  Users  108 registered  2,387 non-registered
  • 68.
  • 69.
    The Project -GeoStor Improve the usability and migrate to the cloud.  300 vector datasets migrated to PostGIS and AWS S3  3TB of raster data  4TB of historical raster data to AWS Glacier
  • 70.
  • 71.
  • 72.
    Benefits of Cloud Stability – Fault tolerant data storage and Safe monitor and support FME Cloud architecture.  Security – Leverage AWS compliance and FME Cloud security policies.  Simplicity – Focus on problem not the administration  Price – 3 times cheaper than on-premises
  • 73.
    Option 1 –On Site Upgrade Monthly Yearly Up-Front DIS Server Space $3,200.00 FME Dekstop $6,000.00 FME Server $12,000.00 Dell Hardware $100,000.00 SQL Server Windows Server Std (Software Assurance)[5] $595.00 Windows Sevrer Enterprise (Software Assurance) [1] $385.00 Microsoft SQL Server (Software Assurance) $270.00 Microsoft SQL Server Standard Edition (Licenses) $294.00 Microsoft Server Std Edition (License) [6] $1,662.00 Microsoft Windows Server Enterprise Ed (Licences) [5] $4,490.00 Symantec $680.00 Tape Backups (No server) $2,000.00 Total Recurring Montly Payment $3,200.00 Total Recurring Yearly Payment $26,376.00 Total One Time Cost (Every 3 yrs) $102,000.00 Total Three Year Cost $296,328.00 True Monthly Cost (/36 months) $8,231.33 Intangiable Costs Hardware Maintenance Time DIS Process Non-Scaleable
  • 74.
    Option 2 –Cloud Monthly Yearly Up-Front FME Dekstop $6,000.00 FME Cloud $14,400.00 EC2 Instance (m3.large) $395.81 AWS Storage (EBS 780GB) $77.11 AWS Storage (S3 2TB) $61.30 AWS Storage (Glacier 4.2 TB) $42.41 Total Recurring Montly Payment $576.63 Total Recurring Yearly Payment $20,400.00 Total Three Year Cost $81,958.68 True Monthly Cost (/36 months) $2,276.63 Our rack space costs (real estate on our data center floor) $3,800 per month. Add to that the hardware costs, etc and you start to see why moving to the cloud was a no brainer for us. Anthony Davis, State Arkansas
  • 75.