Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
CONNECT. TRANSFORM. AUTOMATE.
Lizard Island: On Location
FME Stories From Around the World
Iowa, USA
Snow Plows, ArcGIS Online, an
iPhone, and FME
Snow Plows, ArcGIS Online, an
iPhone, and FME
 Eric Abrams, Iowa Department of Transportation
 901 Snow Plows
 32 – 34 ...
Workflow
Automatic Vehicle Location
 Invisible to driver
 Real-time flow of data
– position, status,
material usage,
conditions
...
Dashcams
 Dash-mounted
iPhones send image
stream when
vehicle is in motion
 FME handles KML
generation and
upload to Win...
AGOL Public Data Access
 ArcGIS Online
keeps public
informed with
current plow status
and conditions
Dashcam Feeds
 Public can also see
what the driver is
seeing for better
awareness of road
and weather
conditions – making...
Safer Winter Driving
Sharing Open Data on GitHub
with FME
La Rioja, Spain
Sharing Public Data
 Ide Rioja committed to sharing and
collaborating on public data.
 Spatial Data Sharing taken to the...
Why GitHub
 GitHub is a web-based Version Control System
(VCS) which records changes to a file or set of
files over time....
Why GitHub
Sharing Public Data
How does FME Help?
 Of course FME translates data from Oracle
Spatial to GeoJSON for GitHub
 But first!
 FME reads the ...
How does FME Help?
The Beauty of GeoJSON in GitHub
 GitHub supports automatic rendering of
GeoJSON repositories using Leaflet.js
 Looking a...
Learn More at FME User Conference
 Extended Version of this topic will be
presented at the FME International User
Confere...
CC BY-SA 3.0 Tony Nordin
FME Server and the
Gävle Data Portal
Gävle, Sweden
FME Server and the Gävle Data
Portal
 Peter Jäderkvist, GIS
Developer & FME Certified
Professional, Community
Developmen...
Various Maps to DWG
 Most popular workspace
 Map type (5), contours,
metadata, AutoCAD version
 XML geometry + paramete...
Specialty DWG Requests
 Custom workspaces
generate specialty DWG
output for other users
 Water & sewer mains for
local w...
3D Model to PDF, Sketchup or
DWG
 Output: Sketchup 8, 3D-PDF and DWG
 Add streets and water, yes/no
 Drape roof tops wi...
1. all parameters set to yes
except for “add roof models”.
2. all parameters set to no.
3. streets water and roof models
s...
Reprojection Services
 On-demand coordinate reprojection
 File reprojection with error checking and format conversion
FME Portal Job History
 Job details
written to history
database
 Over 2100 run
since launch
 Web app shows
usage
Linear Referencing and
Pipe Video with FME
Los Altos, CA, USA
Linear Referencing and Pipe
Video with FME
 Amanda Graf and Raymond Kinser, FME
Certified Professionals, California CAD S...
Data QA Issues
 No spatial coordinates or geometry
 Data inconsistencies across video data vendors
and databases
 Diffe...
Data Cleanup &
Homogenization
 Filter for unwanted and bad
data
 Time stamp formatting
 Defect notation
standardization...
Geometry Creation
 Adjust video session data for
best pipe length
 Adjust for directionality
(video with/against flow)
...
Video Data Sharing
Results
 Easy access to data for all
 Future processing of new
observation video
automated
 City saves money on
future ...
Tableau Dataset
Creation
Birmingham, UK
Tableau Dataset Creation
 Dami Sonoiki, FME Certified
Professional, Dotted Eyes
(Miso)
 Problem: Tableau does great
data...
Geometry Manipulation
 Separate individual boundaries with Deaggregator
 Generalize and reduce vertices
 Deal with donu...
String Manipulation
 Format WKT values to
extract coordinate strings
 StringConcatenator
appends _part_number
supplied b...
List Creation
 ListExploder and
ListIndexer
create x,y coords
for each polygon
 Tableau-ready
format
Address Point
Frontage
Movement
Australia
Address Point Frontage
Movement
 Rajesh Dhull, FME Certified
Professional & Senior Data
Engineer, Data Development
Asia –...
Requirements
 Close to 14 million address points need to be moved to a
new position – property frontage.
 The process sh...
The Approach
1. Create state-wise views in oracle as handling 14
million records in 1 process is not desirable.
2. Create ...
FME Workflow Overview
Filtered,
Buffered
Roads
Lot
Boundary
Polygons
Candidates for movement Pull address centroids from O...
Output
 14 Million
points
processed
each quarter,
automatically.
Railway Platform
Profiling
Brno, Czech Republic
Photo Credit: Roman Báča, CSmap
Railway Platform Profiling
 Rudolf Stastny, FME Certified Professional,
CSmap, s.r.o.
 Challenge: Process hundreds of ra...
Platform Profiles
Convex Hull
Point Selection
DXF Result
Telco Spatial Data Portal
United Arab Emirates
Telco Spatial Data Portal
Telco Spatial Data Portal
 Business Requirement: A Telecom customer
wanted a web portal for secure internal data
sharing/...
System Architecture
Secure access
ArcGIS
Server
Geodata
base
FME
Download
Page
FME
Server
Firewall
Users
Secure Interface
FME Server Processing
 Single workspace with Custom Transformers
 Geodatabase Reader
 Bounding Box
 create
 Clip
 Wr...
Budapest, Hungary
Laser Scanning Roads
and FME
Laser Scanning Roads and FME
 Gyula Sz. Fekete, Head of GIS Development and Data
Production, BKK Közút Zrt. (a company of...
MLS Mission Management
 Aim
 visualize MLS (Mobile Laser Scanning) trajectories
 locations of MLS projects
 attribute ...
Source Data
 MLS Trajectory Data
 Riegl MLS Project Log File
 Image Headers
 Post Processing Workflow Tasks
Output
 GDB with -
 Trajectory information
 Image information
 Post-processing
workflow tasks
 WebGIS GUI with
editab...
CAD to 3D Geodatabase
Conversion
 DGN + MDB
 Tables linked to DGN geometry
CAD to 3D Geodatabase
Conversion
Point Cloud Analysis
 Find road surface
errors based on MLS
scanned 3D point
clouds
 Generate vector data
to be used for...
Analyze & Write: Road Surface
Errors
Analyze & Write: Rugs
CC BY-SA Juhanson
Belgium
Comprehensive
Waterways Data QA
Comprehensive Waterways
Data QA
 Rob Vangeneugden, FME Certified Professional,
GIM nv
 Challenge: Simplify and automate ...
The Project
 Spans multiple waterways
authorities
 3 primary workspaces
 9 embedded custom
transformers
 27 linked cus...
Custom Transformers
 Uses both linked and embedded
 Each data type has a specific custom transformer that identifies
Upd...
Using Parameters
 Published –
 Passed by Django application
 Format-specific (3-GML, 9-shape) and user
credentials, ver...
Database Updates
Example update database process (lock chamber)
Benefits
 Central storage
 Divided management
 Detailed validation feedback
 Summary table
 Geographic files (downloa...
Thank You!
 Questions?
 For more information:
 www.safe.com
 blog.safe.com
Upcoming SlideShare
Loading in …5
×

FME User Stories from Around the World

2,386 views

Published on

See more FME World Tour 2014 presentations at www.safe.com/recap2014

Published in: Technology
  • Be the first to comment

FME User Stories from Around the World

  1. 1. CONNECT. TRANSFORM. AUTOMATE. Lizard Island: On Location FME Stories From Around the World
  2. 2. Iowa, USA Snow Plows, ArcGIS Online, an iPhone, and FME
  3. 3. Snow Plows, ArcGIS Online, an iPhone, and FME  Eric Abrams, Iowa Department of Transportation  901 Snow Plows  32 – 34 inches of snow  9400 miles of road  15 million gallons of brine  120,000 tons of salt on 45 inches of snowfall in 2013  ROI - Every dollar spent on AVL returns $6.40  A 10% reduction of salt is $1.4 million dollars in savings
  4. 4. Workflow
  5. 5. Automatic Vehicle Location  Invisible to driver  Real-time flow of data – position, status, material usage, conditions  Data uploading to Amazon cloud  FME moves data to Oracle Spatial for internal usage, then to AGOL for public viewing
  6. 6. Dashcams  Dash-mounted iPhones send image stream when vehicle is in motion  FME handles KML generation and upload to Windows Azure
  7. 7. AGOL Public Data Access  ArcGIS Online keeps public informed with current plow status and conditions
  8. 8. Dashcam Feeds  Public can also see what the driver is seeing for better awareness of road and weather conditions – making winter driving safer.
  9. 9. Safer Winter Driving
  10. 10. Sharing Open Data on GitHub with FME La Rioja, Spain
  11. 11. Sharing Public Data  Ide Rioja committed to sharing and collaborating on public data.  Spatial Data Sharing taken to the next level  Creative Commons License  Enter GitHub
  12. 12. Why GitHub  GitHub is a web-based Version Control System (VCS) which records changes to a file or set of files over time.  Allows:  commit files to a public repository  revert files back to a previous state  review changes made over time  see who last modified something, and more...
  13. 13. Why GitHub
  14. 14. Sharing Public Data
  15. 15. How does FME Help?  Of course FME translates data from Oracle Spatial to GeoJSON for GitHub  But first!  FME reads the layer list from GitHub using Python Scripted Parameter – git pull  And after!  FME commits updated GeoJSON to GitHub in Shut Down Script – git push  Scheduled Job on FME Server
  16. 16. How does FME Help?
  17. 17. The Beauty of GeoJSON in GitHub  GitHub supports automatic rendering of GeoJSON repositories using Leaflet.js  Looking ahead  geojson.io a Chrome extension for editing  IDE Rioja plans open collaboration on spatial data with GitHub  FME can include links to image data when writing GeoJSON (automatic download service)
  18. 18. Learn More at FME User Conference  Extended Version of this topic will be presented at the FME International User Conference
  19. 19. CC BY-SA 3.0 Tony Nordin FME Server and the Gävle Data Portal Gävle, Sweden
  20. 20. FME Server and the Gävle Data Portal  Peter Jäderkvist, GIS Developer & FME Certified Professional, Community Development Gävle  Provides services and centralized workspace organization, FME usage tracking  Dynamic forms via client communication with FME Server REST API An evolving UI: Peter recently added upload an irregular polygon to clip
  21. 21. Various Maps to DWG  Most popular workspace  Map type (5), contours, metadata, AutoCAD version  XML geometry + parameters triggers FeatureReaders  SchemaMapper, clip & output Example output DWG basemap
  22. 22. Specialty DWG Requests  Custom workspaces generate specialty DWG output for other users  Water & sewer mains for local water company  Power distribution grid for local provider
  23. 23. 3D Model to PDF, Sketchup or DWG  Output: Sketchup 8, 3D-PDF and DWG  Add streets and water, yes/no  Drape roof tops with aerial photography, yes/no  Drape elevation model with aerial photography, yes/no  Add roof models if they exist, yes/no  Some buildings don’t have heights, a parameter decides how to treat those, e.g. “extrude by 7 meters”
  24. 24. 1. all parameters set to yes except for “add roof models”. 2. all parameters set to no. 3. streets water and roof models set to yes Example output sketchup files 1 2 3
  25. 25. Reprojection Services  On-demand coordinate reprojection  File reprojection with error checking and format conversion
  26. 26. FME Portal Job History  Job details written to history database  Over 2100 run since launch  Web app shows usage
  27. 27. Linear Referencing and Pipe Video with FME Los Altos, CA, USA
  28. 28. Linear Referencing and Pipe Video with FME  Amanda Graf and Raymond Kinser, FME Certified Professionals, California CAD Solutions  Challenge: Map and share non-spatial inspection video footage of all sewer lines for the City of Los Altos.  Approach: Use FME and linear referencing methods to QA and position video, creating an automated, repeatable process.
  29. 29. Data QA Issues  No spatial coordinates or geometry  Data inconsistencies across video data vendors and databases  Differences between measured pipe lengths from the vendors and the City  Inconsistent data entry of defect types
  30. 30. Data Cleanup & Homogenization  Filter for unwanted and bad data  Time stamp formatting  Defect notation standardization  Match to best known good City records  QA for flow direction  Catch issues for manual intervention
  31. 31. Geometry Creation  Adjust video session data for best pipe length  Adjust for directionality (video with/against flow)  Create geometry using linear measures, chopper, and NeighborFinder
  32. 32. Video Data Sharing
  33. 33. Results  Easy access to data for all  Future processing of new observation video automated  City saves money on future contracts
  34. 34. Tableau Dataset Creation Birmingham, UK
  35. 35. Tableau Dataset Creation  Dami Sonoiki, FME Certified Professional, Dotted Eyes (Miso)  Problem: Tableau does great data visualization, but lacks good mapping capabilities  Solution: Use FME to break down polygons
  36. 36. Geometry Manipulation  Separate individual boundaries with Deaggregator  Generalize and reduce vertices  Deal with donuts  Produce OGC Well Known Text values for polys
  37. 37. String Manipulation  Format WKT values to extract coordinate strings  StringConcatenator appends _part_number supplied by Deaggregator with Code_Count to provide unique ID PolygonPart  PolygonPart defines Detail for Tableau reconstruction
  38. 38. List Creation  ListExploder and ListIndexer create x,y coords for each polygon  Tableau-ready format
  39. 39. Address Point Frontage Movement Australia
  40. 40. Address Point Frontage Movement  Rajesh Dhull, FME Certified Professional & Senior Data Engineer, Data Development Asia – Pacific, Pitney Bowes Software  Problem: Addresses are pinpointed by lot centroids, but services are provided at the street.  Solution: Create a value- added, dynamic geocoding dataset with addresses located at the front of the property. It’s (almost) always best that your taxi arrives at the front door rather than the living room.
  41. 41. Requirements  Close to 14 million address points need to be moved to a new position – property frontage.  The process should be robust, reliable and repeatable every quarter.  The process should be able to handle heavy datasets.  The process should be able to fit in the existing processes smoothly and should not lead to extra times or delays in the product releases.  Source address points in Oracle, referential data (boundaries, streets) in MapInfo Tab files
  42. 42. The Approach 1. Create state-wise views in oracle as handling 14 million records in 1 process is not desirable. 2. Create single FME workspace for frontage movement process for states with smaller datasets. 3. Split this process in smaller manageable processes for states with bigger datasets as FME performance varies greatly based on the size of the datasets.
  43. 43. FME Workflow Overview Filtered, Buffered Roads Lot Boundary Polygons Candidates for movement Pull address centroids from Oracle Update Oracle
  44. 44. Output  14 Million points processed each quarter, automatically.
  45. 45. Railway Platform Profiling Brno, Czech Republic Photo Credit: Roman Báča, CSmap
  46. 46. Railway Platform Profiling  Rudolf Stastny, FME Certified Professional, CSmap, s.r.o.  Challenge: Process hundreds of railway platform profile DXF files derived from laser scans to look for areas outside tolerances (preventing collisions)  Solution: Automate it with FME
  47. 47. Platform Profiles
  48. 48. Convex Hull
  49. 49. Point Selection
  50. 50. DXF Result
  51. 51. Telco Spatial Data Portal United Arab Emirates Telco Spatial Data Portal
  52. 52. Telco Spatial Data Portal  Business Requirement: A Telecom customer wanted a web portal for secure internal data sharing/downloading.  Layer and coordinate system selections needed  Sources included imagery, vectors, and proprietary data, updated daily by external contractors
  53. 53. System Architecture Secure access ArcGIS Server Geodata base FME Download Page FME Server Firewall Users
  54. 54. Secure Interface
  55. 55. FME Server Processing  Single workspace with Custom Transformers  Geodatabase Reader  Bounding Box  create  Clip  Write to choice of format and projection
  56. 56. Budapest, Hungary Laser Scanning Roads and FME
  57. 57. Laser Scanning Roads and FME  Gyula Sz. Fekete, Head of GIS Development and Data Production, BKK Közút Zrt. (a company of the Municipality of Budapest) 1. Management of Mobile Laser Scanning (MLS) missions and post- processing 2. Data conversion from CAD-based data capture system to Oracle ArcSDE GDB 3. Point cloud data analysis
  58. 58. MLS Mission Management  Aim  visualize MLS (Mobile Laser Scanning) trajectories  locations of MLS projects  attribute information of each scanning project – project metadata  (scanner, driver, acquisition time, etc.)  positions of all exposed images  attribute information of each images comes from image header information.  provide a GDB where post-processing steps can be visualized and modified on a WebGIS GUI.
  59. 59. Source Data  MLS Trajectory Data  Riegl MLS Project Log File  Image Headers  Post Processing Workflow Tasks
  60. 60. Output  GDB with -  Trajectory information  Image information  Post-processing workflow tasks  WebGIS GUI with editable workflow tasks
  61. 61. CAD to 3D Geodatabase Conversion  DGN + MDB  Tables linked to DGN geometry
  62. 62. CAD to 3D Geodatabase Conversion
  63. 63. Point Cloud Analysis  Find road surface errors based on MLS scanned 3D point clouds  Generate vector data to be used for further spatial analysis  Read: 3D point clouds and parameters
  64. 64. Analyze & Write: Road Surface Errors
  65. 65. Analyze & Write: Rugs
  66. 66. CC BY-SA Juhanson Belgium Comprehensive Waterways Data QA
  67. 67. Comprehensive Waterways Data QA  Rob Vangeneugden, FME Certified Professional, GIM nv  Challenge: Simplify and automate a complex data validation process for waterways authorities  Solution: Create a Django user interface and use FME Server to validate, manage results, and perform database updates
  68. 68. The Project  Spans multiple waterways authorities  3 primary workspaces  9 embedded custom transformers  27 linked custom transformers  +/- 3000 transformers
  69. 69. Custom Transformers  Uses both linked and embedded  Each data type has a specific custom transformer that identifies Update/Insert/Delete by comparing geometry and attributes to operational tables:  Terminal  Bridge  Bollard  Fairway  Berth  Node  Lock  Lock Chamber
  70. 70. Using Parameters  Published –  Passed by Django application  Format-specific (3-GML, 9-shape) and user credentials, verify authority  Private –  PostGIS connections  Schema parameters  Output location
  71. 71. Database Updates Example update database process (lock chamber)
  72. 72. Benefits  Central storage  Divided management  Detailed validation feedback  Summary table  Geographic files (downloads)  Full update history  Easy to expand (data formats, validation rules…)  Fast data update / Up-to-date database
  73. 73. Thank You!  Questions?  For more information:  www.safe.com  blog.safe.com

×