FME Around the World


Published on

Lizard Island: On Location
FME Stories From Around the World

Published in: Software
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

FME Around the World

  1. 1. CONNECT. TRANSFORM. AUTOMATE. Lizard Island: On Location FME Stories From Around the World
  2. 2. CONNECT. TRANSFORM. AUTOMATE. Iowa, USA Snow Plows, ArcGIS Online, an iPhone, and FME
  3. 3. CONNECT. TRANSFORM. AUTOMATE. Snow Plows, ArcGIS Online, an iPhone, and FME  Eric Abrams, Iowa Department of Transportation  901 Snow Plows  32 – 34 inches of snow  9400 miles of road  15 million gallons of brine  120,000 tons of salt on 45 inches of snowfall in 2013  ROI - Every dollar spent on AVL returns $6.40  A 10% reduction of salt is $1.4 million dollars in savings
  5. 5. CONNECT. TRANSFORM. AUTOMATE. Automatic Vehicle Location  Invisible to driver  Real-time flow of data – position, status, material usage, conditions  Data uploading to Amazon cloud  FME moves data to Oracle Spatial for internal usage, then to AGOL for public viewing
  6. 6. CONNECT. TRANSFORM. AUTOMATE. Dashcams  Dash-mounted iPhones send image stream when vehicle is in motion  FME handles KML generation and upload to Windows Azure
  7. 7. CONNECT. TRANSFORM. AUTOMATE. AGOL Public Data Access  ArcGIS Online keeps public informed with current plow status and conditions
  8. 8. CONNECT. TRANSFORM. AUTOMATE. Dashcam Feeds  Public can also see what the driver is seeing for better awareness of road and weather conditions – making winter driving safer.
  9. 9. CONNECT. TRANSFORM. AUTOMATE. Safer Winter Driving
  10. 10. CONNECT. TRANSFORM. AUTOMATE. Sharing Open Data on GitHub with FME La Rioja, Spain
  11. 11. CONNECT. TRANSFORM. AUTOMATE. Sharing Public Data  Ide Rioja committed to sharing and collaborating on public data.  Spatial Data Sharing taken to the next level  Creative Commons License  Enter GitHub
  12. 12. CONNECT. TRANSFORM. AUTOMATE. Why GitHub  GitHub is a web-based Version Control System (VCS) which records changes to a file or set of files over time.  Allows:  commit files to a public repository  revert files back to a previous state  review changes made over time  see who last modified something, and more...
  14. 14. CONNECT. TRANSFORM. AUTOMATE. Sharing Public Data
  15. 15. CONNECT. TRANSFORM. AUTOMATE. How does FME Help?  Of course FME translates data from Oracle Spatial to GeoJSON for GitHub  But first!  FME reads the layer list from GitHub using Python Scripted Parameter – git pull  And after!  FME commits updated GeoJSON to GitHub in Shut Down Script – git push  Scheduled Job on FME Server
  16. 16. CONNECT. TRANSFORM. AUTOMATE. How does FME Help?
  17. 17. CONNECT. TRANSFORM. AUTOMATE. The Beauty of GeoJSON in GitHub  GitHub supports automatic rendering of GeoJSON repositories using Leaflet.js  Looking ahead  a Chrome extension for editing  IDE Rioja plans open collaboration on spatial data with GitHub  FME can include links to image data when writing GeoJSON (automatic download service)
  18. 18. CONNECT. TRANSFORM. AUTOMATE. Learn More at FME User Conference  Extended Version of this topic will be presented at the FME International User Conference
  19. 19. CONNECT. TRANSFORM. AUTOMATE.CC BY-SA 3.0 Tony Nordin FME Server and the Gävle Data Portal Gävle, Sweden
  20. 20. CONNECT. TRANSFORM. AUTOMATE. FME Server and the Gävle Data Portal  Peter Jäderkvist, GIS Developer & FME Certified Professional, Community Development Gävle  Provides services and centralized workspace organization, FME usage tracking  Dynamic forms via client communication with FME Server REST API An evolving UI: Peter recently added upload an irregular polygon to clip
  21. 21. CONNECT. TRANSFORM. AUTOMATE. Various Maps to DWG  Most popular workspace  Map type (5), contours, metadata, AutoCAD version  XML geometry + parameters triggers FeatureReaders  SchemaMapper, clip & output Example output DWG basemap
  22. 22. CONNECT. TRANSFORM. AUTOMATE. Specialty DWG Requests  Custom workspaces generate specialty DWG output for other users  Water & sewer mains for local water company  Power distribution grid for local provider
  23. 23. CONNECT. TRANSFORM. AUTOMATE. 3D Model to PDF, Sketchup or DWG  Output: Sketchup 8, 3D-PDF and DWG  Add streets and water, yes/no  Drape roof tops with aerial photography, yes/no  Drape elevation model with aerial photography, yes/no  Add roof models if they exist, yes/no  Some buildings don’t have heights, a parameter decides how to treat those, e.g. “extrude by 7 meters”
  24. 24. CONNECT. TRANSFORM. AUTOMATE. 1. all parameters set to yes except for “add roof models”. 2. all parameters set to no. 3. streets water and roof models set to yes Example output sketchup files 1 2 3
  25. 25. CONNECT. TRANSFORM. AUTOMATE. Reprojection Services  On-demand coordinate reprojection  File reprojection with error checking and format conversion
  26. 26. CONNECT. TRANSFORM. AUTOMATE. FME Portal Job History  Job details written to history database  Over 2100 run since launch  Web app shows usage
  27. 27. CONNECT. TRANSFORM. AUTOMATE. Linear Referencing and Pipe Video with FME Los Altos, CA, USA
  28. 28. CONNECT. TRANSFORM. AUTOMATE. Linear Referencing and Pipe Video with FME  Amanda Graf and Raymond Kinser, FME Certified Professionals, California CAD Solutions  Challenge: Map and share non-spatial inspection video footage of all sewer lines for the City of Los Altos.  Approach: Use FME and linear referencing methods to QA and position video, creating an automated, repeatable process.
  29. 29. CONNECT. TRANSFORM. AUTOMATE. Data QA Issues  No spatial coordinates or geometry  Data inconsistencies across video data vendors and databases  Differences between measured pipe lengths from the vendors and the City  Inconsistent data entry of defect types
  30. 30. CONNECT. TRANSFORM. AUTOMATE. Data Cleanup & Homogenization  Filter for unwanted and bad data  Time stamp formatting  Defect notation standardization  Match to best known good City records  QA for flow direction  Catch issues for manual intervention
  31. 31. CONNECT. TRANSFORM. AUTOMATE. Geometry Creation  Adjust video session data for best pipe length  Adjust for directionality (video with/against flow)  Create geometry using linear measures, chopper, and NeighborFinder
  32. 32. CONNECT. TRANSFORM. AUTOMATE. Video Data Sharing
  33. 33. CONNECT. TRANSFORM. AUTOMATE. Results  Easy access to data for all  Future processing of new observation video automated  City saves money on future contracts
  34. 34. CONNECT. TRANSFORM. AUTOMATE. Tableau Dataset Creation Birmingham, UK
  35. 35. CONNECT. TRANSFORM. AUTOMATE. Tableau Dataset Creation  Dami Sonoiki, FME Certified Professional, Dotted Eyes (Miso)  Problem: Tableau does great data visualization, but lacks good mapping capabilities  Solution: Use FME to break down polygons
  36. 36. CONNECT. TRANSFORM. AUTOMATE. Geometry Manipulation  Separate individual boundaries with Deaggregator  Generalize and reduce vertices  Deal with donuts  Produce OGC Well Known Text values for polys
  37. 37. CONNECT. TRANSFORM. AUTOMATE. String Manipulation  Format WKT values to extract coordinate strings  StringConcatenator appends _part_number supplied by Deaggregator with Code_Count to provide unique ID PolygonPart  PolygonPart defines Detail for Tableau reconstruction
  38. 38. CONNECT. TRANSFORM. AUTOMATE. List Creation  ListExploder and ListIndexer create x,y coords for each polygon  Tableau-ready format
  39. 39. CONNECT. TRANSFORM. AUTOMATE. Address Point Frontage Movement Australia
  40. 40. CONNECT. TRANSFORM. AUTOMATE. Address Point Frontage Movement  Rajesh Dhull, FME Certified Professional & Senior Data Engineer, Data Development Asia – Pacific, Pitney Bowes Software  Problem: Addresses are pinpointed by lot centroids, but services are provided at the street.  Solution: Create a value- added, dynamic geocoding dataset with addresses located at the front of the property. It’s (almost) always best that your taxi arrives at the front door rather than the living room.
  41. 41. CONNECT. TRANSFORM. AUTOMATE. Requirements  Close to 14 million address points need to be moved to a new position – property frontage.  The process should be robust, reliable and repeatable every quarter.  The process should be able to handle heavy datasets.  The process should be able to fit in the existing processes smoothly and should not lead to extra times or delays in the product releases.  Source address points in Oracle, referential data (boundaries, streets) in MapInfo Tab files
  42. 42. CONNECT. TRANSFORM. AUTOMATE. The Approach 1. Create state-wise views in oracle as handling 14 million records in 1 process is not desirable. 2. Create single FME workspace for frontage movement process for states with smaller datasets. 3. Split this process in smaller manageable processes for states with bigger datasets as FME performance varies greatly based on the size of the datasets.
  43. 43. CONNECT. TRANSFORM. AUTOMATE. FME Workflow Overview Filtered, Buffered Roads Lot Boundary Polygons Candidates for movement Pull address centroids from Oracle Update Oracle
  44. 44. CONNECT. TRANSFORM. AUTOMATE. Output  14 Million points processed each quarter, automatically.
  45. 45. CONNECT. TRANSFORM. AUTOMATE. Railway Platform Profiling Brno, Czech Republic Photo Credit: Roman Báča, CSmap
  46. 46. CONNECT. TRANSFORM. AUTOMATE. Railway Platform Profiling  Rudolf Stastny, FME Certified Professional, CSmap, s.r.o.  Challenge: Process hundreds of railway platform profile DXF files derived from laser scans to look for areas outside tolerances (preventing collisions)  Solution: Automate it with FME
  47. 47. CONNECT. TRANSFORM. AUTOMATE. Platform Profiles
  49. 49. CONNECT. TRANSFORM. AUTOMATE. Point Selection
  51. 51. CONNECT. TRANSFORM. AUTOMATE. Telco Spatial Data Portal United Arab Emirates Telco Spatial Data Portal
  52. 52. CONNECT. TRANSFORM. AUTOMATE. Telco Spatial Data Portal  Business Requirement: A Telecom customer wanted a web portal for secure internal data sharing/downloading.  Layer and coordinate system selections needed  Sources included imagery, vectors, and proprietary data, updated daily by external contractors
  53. 53. CONNECT. TRANSFORM. AUTOMATE. System Architecture Secure access ArcGIS Server Geodata base FME Download Page FME Server Firewall Users
  54. 54. CONNECT. TRANSFORM. AUTOMATE. Secure Interface
  55. 55. CONNECT. TRANSFORM. AUTOMATE. FME Server Processing  Single workspace with Custom Transformers  Geodatabase Reader  Bounding Box  create  Clip  Write to choice of format and projection
  56. 56. CONNECT. TRANSFORM. AUTOMATE. Budapest, Hungary Laser Scanning Roads and FME
  57. 57. CONNECT. TRANSFORM. AUTOMATE. Laser Scanning Roads and FME  Gyula Sz. Fekete, Head of GIS Development and Data Production, BKK Közút Zrt. (a company of the Municipality of Budapest) 1. Management of Mobile Laser Scanning (MLS) missions and post-processing 2. Data conversion from CAD-based data capture system to Oracle ArcSDE GDB 3. Point cloud data analysis
  58. 58. CONNECT. TRANSFORM. AUTOMATE. MLS Mission Management  Aim  visualize MLS (Mobile Laser Scanning) trajectories  locations of MLS projects  attribute information of each scanning project – project metadata  (scanner, driver, acquisition time, etc.)  positions of all exposed images  attribute information of each images comes from image header information.  provide a GDB where post-processing steps can be visualized and modified on a WebGIS GUI.
  59. 59. CONNECT. TRANSFORM. AUTOMATE. Source Data  MLS Trajectory Data  Riegl MLS Project Log File  Image Headers  Post Processing Workflow Tasks
  60. 60. CONNECT. TRANSFORM. AUTOMATE. Output  GDB with -  Trajectory information  Image information  Post-processing workflow tasks  WebGIS GUI with editable workflow tasks
  61. 61. CONNECT. TRANSFORM. AUTOMATE. CAD to 3D Geodatabase Conversion  DGN + MDB  Tables linked to DGN geometry
  62. 62. CONNECT. TRANSFORM. AUTOMATE. CAD to 3D Geodatabase Conversion
  63. 63. CONNECT. TRANSFORM. AUTOMATE. Point Cloud Analysis  Find road surface errors based on MLS scanned 3D point clouds  Generate vector data to be used for further spatial analysis  Read: 3D point clouds and parameters
  64. 64. CONNECT. TRANSFORM. AUTOMATE. Analyze & Write: Road Surface Errors
  65. 65. CONNECT. TRANSFORM. AUTOMATE. Analyze & Write: Rugs
  66. 66. CONNECT. TRANSFORM. AUTOMATE.CC BY-SA Juhanson Belgium Comprehensive Waterways Data QA
  67. 67. CONNECT. TRANSFORM. AUTOMATE. Comprehensive Waterways Data QA  Rob Vangeneugden, FME Certified Professional, GIM nv  Challenge: Simplify and automate a complex data validation process for waterways authorities  Solution: Create a Django user interface and use FME Server to validate, manage results, and perform database updates
  68. 68. CONNECT. TRANSFORM. AUTOMATE. The Project  Spans multiple waterways authorities  3 primary workspaces  9 embedded custom transformers  27 linked custom transformers  +/- 3000 transformers
  69. 69. CONNECT. TRANSFORM. AUTOMATE. Custom Transformers  Uses both linked and embedded  Each data type has a specific custom transformer that identifies Update/Insert/Delete by comparing geometry and attributes to operational tables:  Terminal  Bridge  Bollard  Fairway  Berth  Node  Lock  Lock Chamber
  70. 70. CONNECT. TRANSFORM. AUTOMATE. Using Parameters  Published –  Passed by Django application  Format-specific (3-GML, 9-shape) and user credentials, verify authority  Private –  PostGIS connections  Schema parameters  Output location
  71. 71. CONNECT. TRANSFORM. AUTOMATE. Database Updates Example update database process (lock chamber)
  72. 72. CONNECT. TRANSFORM. AUTOMATE. Benefits  Central storage  Divided management  Detailed validation feedback  Summary table  Geographic files (downloads)  Full update history  Easy to expand (data formats, validation rules…)  Fast data update / Up-to-date database
  73. 73. CONNECT. TRANSFORM. AUTOMATE. Thank You!  Questions?  For more information:  