FME User Stories from Around the World


Published on

See more FME World Tour 2014 presentations at

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • An Iowa winter – the Airship crew doesn’t mind if the roads aren’t plowed.
  • FME handling inbound live flow of data from plows and constant upload to cloud, ultimately ArcGIS Online – including photo stream from dash-mounted iPhones for public access.Internal portal (WOPR) (Geocortex Essentials)External portal (WOPR Mobile) (Geocortex Essentials)Public Portal (ArcGIS online)Technology UsedOracleESRIIntergraphBentleyLatitude GeographicsFME
  • Vineyards in La Rioja, Spain. Looks like the crew may be taking off after a stop to sample La Rioja’s famous wine….
  • May be a photo update here
  • TheRapidEye image talk was in the 2013 world tour, very advanced use of RasterExpressionEvaluator to process RapidEye imagery for vegetation classificationIn In 2011 at FME World Tour in Madrid Pablo presented their orthphoto PDF Series, included scalebar, grid, map surround etc
  • Forward looking State govt. with a mandate to share dataEmphasize Creative Commons License they retain rights but others can use and share with Attribution,License is: “Attribution 4.0 International“ You can view the license agreement in the GitHub Repository
  • This is perfect for govt. data where we need to track changes, ensure the current version is up to date, push updates, see who changed what when…
  • They are pushing GeoJSON data to GitHub (we’ll talk about GeoJSON later)
  • Of course FME is used to read from Oracle Spatial and write GeoJSONBut interesting that FME is also used to pull and push from GitHubLayers ListThey maintain the layer list in GitHub – these are the layers they want to store up thereWhen the workspace runs a scripted parameter reads the layers list and returns the list into the Feature Types to Reader parameter for the Oracle readerPushing the Dataa shut down TCL script pushed the data in GitHubImage LinksThe GeoJSON features can have http links to imagery downloads which is a super easy way to make a data download service
  • Workspace is in directory
  • If you just click on a GeoJSON file in the repository it automatically shows it in a map window, you can try it here: example here: chrome extension give an editing environment, try it
  • The famous Gavle Goat has been set ablaze at least 24 times since the first one was put on display back in 1966.In 2010, two men tried to bribe the security guards so they could kidnap it with a helicopter. Who knows what the FME Airship crew are up to in this picture!
  • Project description and goal The ultimate goal of the portal was to increase the accessibility to GIS-services in the municipality of Gävle. From the start I have tried to keep the portal as intuitive and simple as possible to use. I have continuously worked on improving it by adding new functionality and simplifying the user interface. The first version only included clip and ship functionality but in the second version I added reprojection services as well. Another very important goal was to simply take full control of the FME use. It’s hard to control and maintain workspaces spread out in a common network file system. Uploading them to a secured FME-server with the FME portal as a single entry point solves this and gives possibility to keep track of usage. All forms are being built dynamically by client side communication with the FME-server REST API. This makes the application really easy to maintain because any workspace changes to parameters will instantly show up in the portals UI. The source code is HTML and JavaScript and the map functionality was added using ArcGIS Server JavaScript API. The FME-server playground has been a very helpful starting point for me in the development process. Security is maintained in Active Directory and the users login using their ordinary Windows credentials.
  • This is the most commonly used workspace in the FME portal. This workspace takes XML geometry and other parameters to trigger a series of FeatureReadersto read a bunch of feature classes from different ArcSDE databases. The features are then mapped and processed using the SchemaMapper, clipped and output as an AutoCadRealDWG. User input parameters (except for the geometry) are: - Map product type with five different options, each triggers different FeatureReaders in the workspace. - Elevation contour lines, yes/no - Purge empty layers, yes/no - Include map declaration document with metadata about the chosen product, yes/no - AutoCad Version multiple versions
  • Creates a 3D-model out of buildings, an elevation model, aerial photography, streets and water polygons. The elevation model is the Swedish National Elevation Model grid with a resolution of 2 meters. Buildings are placed on top of the elevation model and are extruded based on heights in attributes to create box models. The aerial photography, water and streets can optionally be draped on top of the model.The workspace is mainly used by the town planners of Gävle to quickly get good starting materials when planning for new neighborhoods etc
  • Coordinate reprojection service When a reprojection workspace are selected the drawing tools gets disabled and an additional form appear on top of the map. When reprojecting a few coordinates manually, the user simply adds as many rows needed in the form and enters the coordinates. When done, the user clicks “Klar”, the coordinates are then validated before the order button is enabled. A format and a source and destination coordinate system are selected. XML holding each coordinate and ID are built and sent as a parameter to the workspace. File reprojection service The file reprojection workspace supports multiple inputs of both shapefiles and AutoCad DWG files. It is also possible to upload zip-files. This service can also be used to simply convert from shapefiles to DWG or vice versa. To prevent users from making bad reprojections, like entering the wrong coordinate system, the workspace performing the reprojection validates the coordinates of each feature spatially against a predefined extent for each coordinate system. If features are outside the extent, a Terminator transformer is triggered returning the FME portal an error code.
  • Every time a user runs a workspace in the FME portal, the geometry, timestamp, user-id, product type and area are written to a history database table. So far, more than 2100 clip and ship jobs have been completed since the launch. I have created a simple web application where I visually can monitor the usage over time. Clicking on an area shows a popup with information about the date, user-id, product and area.More than 2100 clip and ship jobs completed since the startup in april 2012. ResultsWhen the FME portal was launched 1.5 years ago, initially there were 5-10 users. Today, the portal have about 55-60 returning users. FME has become a crucial system in the daily work for many employees at the municipality of Gävle and a platform for me to spread the power of FME in a very user-friendly way. I am currently working on the 3.0 version.
  • The crew passes over the Maryknoll Residence in Los Altos California, built in 1926 as a residential seminaryStoakes
  • Initial Problem:• Massive investment in Sewer video footage that could not be easily accessed by City personnel• Data collected over multiple years by multiple vendors each using a different database• Sewer System updates and data entry errors of video personnel make matching of all entries a problem• Creating list of sewer pipe segments without video recording• Need to Map specific defect events based on a database description• No spatial data component of the video information – Only designations of distance along a line
  • Major Factors to deal with:• Creating spatial data from non-spatial database containing no coordinate information• Data inconsistencies between multiple vendors and multiple databases• Differences in measured pipe lengths from the video vendor and the data as maintained by the City• Inconsistent data entry on defect type notations
  • Image: part of the QA process standardizes defect type notations. Unifying the data: different vendors had different equipment and approaches.
  • • Match data from City Sewer shape file based on UpStream_DownStream naming convention• Factor distances based on actual pipe length from City data source to more accurately position (i.e. rescale the video location lengths)• Employ Linear Referencing to generate geometry from the combination of a shape file with database distance entries• Generate shape file of desired defect events for publication in the City GIS site
  • City personnel can now access inspection details via the map interface, with video spatially linked.
  • This is a hole in the pipe – and it should probably be fixed.Results:• Comprehensive data inventory of system defects mapped in the GIS allowing for easy linking to the video segment showing the defect• City saved money by being able to select a video vendor based on price of delivery rather than one that matched prior delivery database
  • The crew does a flyby of the iconic Bullring Mall in Birmingham.
  • Business Problem: Tableau is a visualisation tool which helps understand data better. One big challenge with the tool is that it lacks good mapping capabilities especially for the UK. Dotted Eyes needed to create polygons compatible with Tableau Software to supply to her clients. Tableau also works with only WGS84 coordinate system so there was need to convert all the datasets from British Nation Grid to WGS84. Business Solution: Since Tableau does not have capabilities of importing polygons directly into the software, the only option to visualise data as polygons in the software was to break down the polygons into a string of points and used the coordinates of the points to form polygons in Tableau Software. FME Solution: The accompanying workspace is a workspace created to process Lower Super Output Area (LSOA) dataset produced by the Office of National Statistics (ONS). The workspace takes the LSOA dataset in ESRI Shape format, generalises the polygon and then extracts the different vertices of the individual LSOA polygons, defines the path that Tableau uses to draw the polygons and outputs and outputs to a text file using the TextLine writer.
  • The workspace separates each individual LSOA boundary using the Deaggregator transformer. It then generalises each polygon to reduce the vertices using the Generalizer transformer. Doing this improved performance of the datasets in Tableau as performance in dependent on total number of records. Since some of the boundaries may contain donut holes, the DonutBridgeBuilder was used to build connections between donut holes and the outer boundaries. The GeometryExtractor transformer was then used to extract the OGC Well Known Text (WKT) values of the individual polygons.
  • StringReplacer transformers were used to format the extracted well Known Text (WKT) values of the polygons to extract the corresponding coordinate strings of each polygon. The StringConcatenator transformer was first used to append _part_number derived for each polygon from the Deaggregator with the Code_Count attribute to define a unique PolygonPart. This PolygonPart is used to define the Detail which Tableau uses in drawing polygons.
  • The ListExploder and ListIndexer transformers were used to create a list of X and Y coordinates of the points making up each polygons. The resulting LSOA dataset can be found here:
  • select all the property parcel polygons (MapInfo tab file) that are within the non-rural polygon boundaries Filter roads to get rid of things like freeways, service roadsBuffer them according to pre-determined setbacks based on road typeReproject to GDA-94Intersect with lot poly – MapInfo TabIntersected candidates used to pull from Oracle table – appropriate parcel centroid pointsPoints are moved to front of lot (can’t show specific details of this) - Dean: This to me seems to be the main point of the project. Not sure why we cant talk about it or show at least a hi level data flow for it. Looks to me like he isnt moving the points at all but rather finding the centre point and then using the spatial relator to find the closest existing address point read from another Oracle table.Update (not insert) back into Oracle with moved positions.
  • Stoakes:Goal of this project:Take Laser Scanning results in DWG files, as profiles of station platforms. Determine if the new or upgraded platform might interfere with the trains.What’s interesting with this example is that a very similar problem was presented at CARIS hydrographic conference – except in that case they were usingsidescan sonar (underwater point cloud?) to pickup the profiles of docks and:Ensure ships wouldn’t be damaged by docks that were ‘bulging’Determine when docks need repair
  • Project Name: Point Selection Source: DXF Files Profiles of railway platforms from laser scanning. There are several hundreds of profiles from 1 platform. In every DXF file there are several layers. I use only the layer "PROFILE LINE". Destination: DXF file - 3 layers: 1 - original profile line (PROFILE_LINE) 2 - convex hull around selected part of platform (CONVEX_HULL) 3 - 2 result points, one on the top and one on the right side of the platform (POINTS) PNG file - for quick preview of the situation. Contains original profile, convex hull, 2 result points, 1cm grid for better orientation. Image has name of the source file. XLS File - database of 2 result coords (X, Y) for each source file. There are three columns (File - filename, X - result coord. X, Y - result coordinate Y). Goal of the Project: I have to find 2 key coordinates on the platform, that can customer use to make a reclaim to constructor of new platform. If the platform has defects (edges of the platform are outside of selected X and Y bounds), constructor has to make repairs. This control prevents the collision of the train with platform on railway stations. Notes: Project is prepared for Batch translation of hundreds of source DXF files. Batch translation creates one destination DXF file and one PNG image for each source file. Destination XLS file is one for all source files. Writer Mode is set to "INHERIT FROM WRITER", so each new translation creates new rows with XY result coordinates. Included Files: 3 Source DXF Files, FME Workspace, Example output
  • Stoakes:Clip the scan to the platform Edge and create a Convex Hull of the platform edge profile
  • Stoakes:Don’ quite get this step. Looks like he’s selecting the 2 points that represent the Ymax and Xmax of the profile that touches the Convex hull – but there are other criteria based on the 1-3.5cm range. I think this must represent the ‘lip’ of the platform
  • The Telecom customer (Telco) acquires data from three resources: Imagery, Vector, and Telecom data from customer’s own database, which is updated daily by external contractors. They collected the imagery data from ArcGIS online and Vector Data, and migrated the data into an ArcSDEGeodatabase. Telco planned to share this data between departments requiring the communication layers for further analysis. In order to ease the process of transferring the data, Telco decided to create a web portal that would cater to the needs of the entities in downloading the data. The web portal would provide an interface to select the layers and download the output format required in their desired coordinate system. FME web page, the web page provides secure access to authorized people to view imagery and Navteq data, plus the Ability to download the network database.
  • Security, The web page has two level of security:  ?Login page, maintained by an XML file  ?Security page in FME Server The XML file defines the username and password for accessing the login page when the user navigates to the page URL. FME security allows the user to download the data, so despite affording user access to the page, the user will not be able to download the data unless he is registered to FME server, which adds an additional layer of security.
  • The solution contains a single workspace, which is published to a repository in FME Server. The users are provided with few parameters, which are served as published parameters in FME Server. Data is read by GEODATABSE_File base reader. The layers decided by Telco were added in the workspace. There was no requirement of adding more layers into the workspace. The parameter ‘Feature Types to Read’ was converted as a published parameter. Therefore, based on user input, the selected feature will be processed. The main workspace is as (Figure 23). In addition, the main workspace includes a custom transformer named “Creates the Bounding Box”, the contents of this custom transformer is as shown in (Figure 24).
  • The Hungarian Parliament Building is the seat of the National Assembly of Hungary, one of Europe's oldest legislative buildings, a notable landmark of Hungary and a popular tourist destination of Budapest.
  • This project was submitted by GyulaFekete who is responsible for RODIS – the Road Data Information System used by the Municipality of Budapest in Hungary.There are three distinct parts to this workflow:Management of the Mobile Laser Scanning project data – used for display of project data locations and analysisData conversion from CAD based data capture into a 3D GIS conversion (GDB) – based on an Oracle ArcSDEGeodatabase.Point Cloud Analysis – looking for defects in the roadsThis is really interesting. Not sure how he does it but they process the LiDAR and extract the ‘dips’ in the roads as well as what looks like pot holes. They extract the pot hole data as ‘Rugs’ which are contours of the road surface feature.
  • Currently have about 1000 km of road to maintain and 4300km of roads for urban traffic management – I believe this includes minor inner city roads. Much of the data was only available in hardcopy format or in digital format with unknown accuracy and reliability.The requirement was to provide surveying accuracy – around 1-5cm, have it up to date, and provide all necessary attributes available for the daily Workflow management system.They are maintaining the public road network and related features some of which was underground along with the urban rail and tram network and metro linesThe new system had to Provide up-to-date data for the whole cityEasy to expand as new road networks are addedProvide full city coverageEconomical
  • 1 - Source data came from a number of Mobile laser scanners, along with some terrestrial scanners. Also had existing databases maintaining attribute information. - primarily Access MDB
  • Output was to be a geodatabase showing trajectory information from the mobile scanners along with image information and have this presented in a WebGIS GUI allowing for editing by Post Processing staff.Much of the work was done with out of the box software packages and some in house development.Acquisition time 20 daysScanned urban roads 650 kmPost processed 3D pointcloud data 400km
  • 2 - Starting with data in Microstation and attribute information in Access – FME was used to link together features and attributes and produce Geodatabase feature classes fitting the schema defined by the project. Extensive use of the SchemaMapper transformer was made to standardize table and attribute names.They ended up with nearly 200 different feature types and up to 20 different attributes for each feature type.This will be extended to add in the underground utility features.
  • The original drawing files were available in 3D geometry. These were matched to the point cloud data to improve the accuracy and update the features. Then this was transferred into the Geodatabase and consequently is viewable in ArcGIS.3D Feature extraction and mapsstarted at1.5 weeks to process 300 mgrew to 1.5 weeks to process 1,100 mcurrently 1.5 weeks to process 2,500 mProduction speed is accelerating with double speed every 2nd week and they expecting similar progress over the next months
  • 3 – Point cloud analysis was done in an effort to find road surface errors such as cracks and potholes.Acquisition time 20 daysScanned urban roads 650 kmPost processed 3D pointcloud data 400km
  • Presenters: These workspaces are too big to show on screen, but you may want to review them here to see what’s going on. This slide has been hidden. RR: It isn’t possible to get a good enough look at what is in here to understand exactly what they are doing. No workspaces seem to have been provided.
  • So from this kind of point cloud information they were able to identify dips in the road surface along with larger hollow patches. A number of complex FME workspaces are used to do this processing. Point cloud extraction, colouring, converting into vector features, etc.
  • Generating contours from the hollows results in this kind of imagery which clearly identifies pot hole features. I am not sure if something is lost in translation but they refer to these features as “rugs” which we interpreted as something like an “area rug” as opposed to cracks.Stoakes:Rugs: patches in the road, pothole-type things, rather than cracks running across the surface. (like a area rug)Currently this company is fullfilling their Corporate Social Responsibility by hiring hearing impaired employees and matching them with employees who are taught some sign language. This kind of team is working very efficiently together
  • Arenberg Castle- Kasteel van Arenberg is a château in Heverlee close to Leuven in Belgium. It is surrounded by a parkThe first manned gas-filled balloon flight in history took off from the front lawn of the château on November 21, 1783; the balloonist was professor Jan Pieter Minckeleers.[citation needed]
  • Stoakes:Django is a free and open source web application framework, written in Python
  • Stoakes:The goal of the project was to create an application were all Flemish waterway authorities (+/- 10 of them) could validate their data and to create a central database where all valid data was loaded.User can load Shape of GMLEnd User: This database will be the source for numerous other applications in the near future.Three main steps:• Read and validate data • Update database• Clear temporary tables for the next run
  • Some parts of the workspace may seem messy. This is caused by the amount of validation rulesthat uses the same input and because all outputs of these rules go to the same writer. Forexample, the fairway data is used in almost every validation rule.The process steps below are executed partially in parallel but on some moments features are helduntil a certain validation is done before continuing (or stopping) the process.1. Empty temporary tables (to be sure the user starts with a clean sheet)2. Read input3. Validate structure of fields in shapefile4. Round decimals in GML to appropriate length5. Validate if input data is within area of authority of user6. Validate if all obligatory fields are present7. Validate if content is ok based on 6 linked custom transfomers (RISindex101 – 106)8. Check if all above validations were ok (stop/continue process)9. Check for each input feature if it is Insert/Update/Delete. Features can only be deleted ifthey are loaded with attribute “end date” filled.10. Execute 17 consistency validation rules (RISIndex201 – 217) on insert and update featuresplus the features that will not be updated or deleted from the operational database tables.This means that all validation rules are processed on the dataset that would be operationalif the loaded dataset was be added.11. Execute 2 connectivity validation rules (RISIndex301 – 302) on insert and update featuresplus the features that will not be updated or deleted from the operational database tables.This means that all validation rules are processed on the dataset that would be operationalif the loaded dataset was be added.12. Fill error message table if consistency or connectivity fails and create a shapefile with errorzones and error messages.13. Fill temporary tables if consistency and connectivity is ok. The temporary tables hold allinput data when they have passed all validation rules. The data is stored so the user canuse the overview in the user interface to verify which updates will be made before updatingthe operational database tables.14. Create shapefile with all changes that can be made including the corresponding feature outof the current operational table.
  • UID custom transformersEach data type (terminal, bridge, bollard, fairway, berth, node, lock and lock chamber) has its ownUID module. These custom transformers are only executed when all previous validations weresuccessful. These transformers check for every single input feature if it is an Update – Insert –Delete (UID) by comparing the geometry and the attributes with the operational database tables.Linked vs Embedded custom transformersThe workspace contains two types of custom transformers, linked and embedded customtransformers. The embedded custom transformers are embedded because they are really part ofthe workspace. The linked transformers are the validation rules. As they are linked, they can easilybe used by the client in a new, separate workspace to validate in FME workbench already somedata. This series of custom transformers can be extended in the future with extra validation rules.
  • ParametersPublishedThe published parameters are the ones passed by the Django application to the FME workspace inFME Server. For each input dataset a parameters is present, ie. 3 for GML input and 9 for shapefileinput. The GML file “RIS” holds data for different types of data. In the shapefile format it needs tobe split.The parameter “gebruiker” (ie. user) is used to know which user does an update and to verify thearea of authority.PrivateThe private parameters hold the connection parameters of the PostGIS database. Theseparameters are used in all transformers and writers involving the database. In this case thedatabase connection parameters are set in one place.Additionally, there are three parameters for the xsd schemas and one parameter for the outputlocation. This output location will hold the file (error zones or update features) that can bedownloaded via the user interface (FME Server Data Download).
  • This workspace transfers the data out of the temporary tables into the operational tables. Itexecutes the actual Insert/Update/Delete and creates a full history.The workspace is started when the user clicks the “wijzig” button (update button) in the userinterface.Process1. Read temporary table2. Read corresponding feature from operational table (in case of update/delete)3. Create update statement4. Update attribute info and change information (user and date)5. Do update6. Empty temporary table7. Empty validation tables
  • FME User Stories from Around the World

    1. 1. CONNECT. TRANSFORM. AUTOMATE. Lizard Island: On Location FME Stories From Around the World
    2. 2. Iowa, USA Snow Plows, ArcGIS Online, an iPhone, and FME
    3. 3. Snow Plows, ArcGIS Online, an iPhone, and FME  Eric Abrams, Iowa Department of Transportation  901 Snow Plows  32 – 34 inches of snow  9400 miles of road  15 million gallons of brine  120,000 tons of salt on 45 inches of snowfall in 2013  ROI - Every dollar spent on AVL returns $6.40  A 10% reduction of salt is $1.4 million dollars in savings
    4. 4. Workflow
    5. 5. Automatic Vehicle Location  Invisible to driver  Real-time flow of data – position, status, material usage, conditions  Data uploading to Amazon cloud  FME moves data to Oracle Spatial for internal usage, then to AGOL for public viewing
    6. 6. Dashcams  Dash-mounted iPhones send image stream when vehicle is in motion  FME handles KML generation and upload to Windows Azure
    7. 7. AGOL Public Data Access  ArcGIS Online keeps public informed with current plow status and conditions
    8. 8. Dashcam Feeds  Public can also see what the driver is seeing for better awareness of road and weather conditions – making winter driving safer.
    9. 9. Safer Winter Driving
    10. 10. Sharing Open Data on GitHub with FME La Rioja, Spain
    11. 11. Sharing Public Data  Ide Rioja committed to sharing and collaborating on public data.  Spatial Data Sharing taken to the next level  Creative Commons License  Enter GitHub
    12. 12. Why GitHub  GitHub is a web-based Version Control System (VCS) which records changes to a file or set of files over time.  Allows:  commit files to a public repository  revert files back to a previous state  review changes made over time  see who last modified something, and more...
    13. 13. Why GitHub
    14. 14. Sharing Public Data
    15. 15. How does FME Help?  Of course FME translates data from Oracle Spatial to GeoJSON for GitHub  But first!  FME reads the layer list from GitHub using Python Scripted Parameter – git pull  And after!  FME commits updated GeoJSON to GitHub in Shut Down Script – git push  Scheduled Job on FME Server
    16. 16. How does FME Help?
    17. 17. The Beauty of GeoJSON in GitHub  GitHub supports automatic rendering of GeoJSON repositories using Leaflet.js  Looking ahead  a Chrome extension for editing  IDE Rioja plans open collaboration on spatial data with GitHub  FME can include links to image data when writing GeoJSON (automatic download service)
    18. 18. Learn More at FME User Conference  Extended Version of this topic will be presented at the FME International User Conference
    19. 19. CC BY-SA 3.0 Tony Nordin FME Server and the Gävle Data Portal Gävle, Sweden
    20. 20. FME Server and the Gävle Data Portal  Peter Jäderkvist, GIS Developer & FME Certified Professional, Community Development Gävle  Provides services and centralized workspace organization, FME usage tracking  Dynamic forms via client communication with FME Server REST API An evolving UI: Peter recently added upload an irregular polygon to clip
    21. 21. Various Maps to DWG  Most popular workspace  Map type (5), contours, metadata, AutoCAD version  XML geometry + parameters triggers FeatureReaders  SchemaMapper, clip & output Example output DWG basemap
    22. 22. Specialty DWG Requests  Custom workspaces generate specialty DWG output for other users  Water & sewer mains for local water company  Power distribution grid for local provider
    23. 23. 3D Model to PDF, Sketchup or DWG  Output: Sketchup 8, 3D-PDF and DWG  Add streets and water, yes/no  Drape roof tops with aerial photography, yes/no  Drape elevation model with aerial photography, yes/no  Add roof models if they exist, yes/no  Some buildings don’t have heights, a parameter decides how to treat those, e.g. “extrude by 7 meters”
    24. 24. 1. all parameters set to yes except for “add roof models”. 2. all parameters set to no. 3. streets water and roof models set to yes Example output sketchup files 1 2 3
    25. 25. Reprojection Services  On-demand coordinate reprojection  File reprojection with error checking and format conversion
    26. 26. FME Portal Job History  Job details written to history database  Over 2100 run since launch  Web app shows usage
    27. 27. Linear Referencing and Pipe Video with FME Los Altos, CA, USA
    28. 28. Linear Referencing and Pipe Video with FME  Amanda Graf and Raymond Kinser, FME Certified Professionals, California CAD Solutions  Challenge: Map and share non-spatial inspection video footage of all sewer lines for the City of Los Altos.  Approach: Use FME and linear referencing methods to QA and position video, creating an automated, repeatable process.
    29. 29. Data QA Issues  No spatial coordinates or geometry  Data inconsistencies across video data vendors and databases  Differences between measured pipe lengths from the vendors and the City  Inconsistent data entry of defect types
    30. 30. Data Cleanup & Homogenization  Filter for unwanted and bad data  Time stamp formatting  Defect notation standardization  Match to best known good City records  QA for flow direction  Catch issues for manual intervention
    31. 31. Geometry Creation  Adjust video session data for best pipe length  Adjust for directionality (video with/against flow)  Create geometry using linear measures, chopper, and NeighborFinder
    32. 32. Video Data Sharing
    33. 33. Results  Easy access to data for all  Future processing of new observation video automated  City saves money on future contracts
    34. 34. Tableau Dataset Creation Birmingham, UK
    35. 35. Tableau Dataset Creation  Dami Sonoiki, FME Certified Professional, Dotted Eyes (Miso)  Problem: Tableau does great data visualization, but lacks good mapping capabilities  Solution: Use FME to break down polygons
    36. 36. Geometry Manipulation  Separate individual boundaries with Deaggregator  Generalize and reduce vertices  Deal with donuts  Produce OGC Well Known Text values for polys
    37. 37. String Manipulation  Format WKT values to extract coordinate strings  StringConcatenator appends _part_number supplied by Deaggregator with Code_Count to provide unique ID PolygonPart  PolygonPart defines Detail for Tableau reconstruction
    38. 38. List Creation  ListExploder and ListIndexer create x,y coords for each polygon  Tableau-ready format
    39. 39. Address Point Frontage Movement Australia
    40. 40. Address Point Frontage Movement  Rajesh Dhull, FME Certified Professional & Senior Data Engineer, Data Development Asia – Pacific, Pitney Bowes Software  Problem: Addresses are pinpointed by lot centroids, but services are provided at the street.  Solution: Create a value- added, dynamic geocoding dataset with addresses located at the front of the property. It’s (almost) always best that your taxi arrives at the front door rather than the living room.
    41. 41. Requirements  Close to 14 million address points need to be moved to a new position – property frontage.  The process should be robust, reliable and repeatable every quarter.  The process should be able to handle heavy datasets.  The process should be able to fit in the existing processes smoothly and should not lead to extra times or delays in the product releases.  Source address points in Oracle, referential data (boundaries, streets) in MapInfo Tab files
    42. 42. The Approach 1. Create state-wise views in oracle as handling 14 million records in 1 process is not desirable. 2. Create single FME workspace for frontage movement process for states with smaller datasets. 3. Split this process in smaller manageable processes for states with bigger datasets as FME performance varies greatly based on the size of the datasets.
    43. 43. FME Workflow Overview Filtered, Buffered Roads Lot Boundary Polygons Candidates for movement Pull address centroids from Oracle Update Oracle
    44. 44. Output  14 Million points processed each quarter, automatically.
    45. 45. Railway Platform Profiling Brno, Czech Republic Photo Credit: Roman Báča, CSmap
    46. 46. Railway Platform Profiling  Rudolf Stastny, FME Certified Professional, CSmap, s.r.o.  Challenge: Process hundreds of railway platform profile DXF files derived from laser scans to look for areas outside tolerances (preventing collisions)  Solution: Automate it with FME
    47. 47. Platform Profiles
    48. 48. Convex Hull
    49. 49. Point Selection
    50. 50. DXF Result
    51. 51. Telco Spatial Data Portal United Arab Emirates Telco Spatial Data Portal
    52. 52. Telco Spatial Data Portal  Business Requirement: A Telecom customer wanted a web portal for secure internal data sharing/downloading.  Layer and coordinate system selections needed  Sources included imagery, vectors, and proprietary data, updated daily by external contractors
    53. 53. System Architecture Secure access ArcGIS Server Geodata base FME Download Page FME Server Firewall Users
    54. 54. Secure Interface
    55. 55. FME Server Processing  Single workspace with Custom Transformers  Geodatabase Reader  Bounding Box  create  Clip  Write to choice of format and projection
    56. 56. Budapest, Hungary Laser Scanning Roads and FME
    57. 57. Laser Scanning Roads and FME  Gyula Sz. Fekete, Head of GIS Development and Data Production, BKK Közút Zrt. (a company of the Municipality of Budapest) 1. Management of Mobile Laser Scanning (MLS) missions and post- processing 2. Data conversion from CAD-based data capture system to Oracle ArcSDE GDB 3. Point cloud data analysis
    58. 58. MLS Mission Management  Aim  visualize MLS (Mobile Laser Scanning) trajectories  locations of MLS projects  attribute information of each scanning project – project metadata  (scanner, driver, acquisition time, etc.)  positions of all exposed images  attribute information of each images comes from image header information.  provide a GDB where post-processing steps can be visualized and modified on a WebGIS GUI.
    59. 59. Source Data  MLS Trajectory Data  Riegl MLS Project Log File  Image Headers  Post Processing Workflow Tasks
    60. 60. Output  GDB with -  Trajectory information  Image information  Post-processing workflow tasks  WebGIS GUI with editable workflow tasks
    61. 61. CAD to 3D Geodatabase Conversion  DGN + MDB  Tables linked to DGN geometry
    62. 62. CAD to 3D Geodatabase Conversion
    63. 63. Point Cloud Analysis  Find road surface errors based on MLS scanned 3D point clouds  Generate vector data to be used for further spatial analysis  Read: 3D point clouds and parameters
    64. 64. Analyze & Write: Road Surface Errors
    65. 65. Analyze & Write: Rugs
    66. 66. CC BY-SA Juhanson Belgium Comprehensive Waterways Data QA
    67. 67. Comprehensive Waterways Data QA  Rob Vangeneugden, FME Certified Professional, GIM nv  Challenge: Simplify and automate a complex data validation process for waterways authorities  Solution: Create a Django user interface and use FME Server to validate, manage results, and perform database updates
    68. 68. The Project  Spans multiple waterways authorities  3 primary workspaces  9 embedded custom transformers  27 linked custom transformers  +/- 3000 transformers
    69. 69. Custom Transformers  Uses both linked and embedded  Each data type has a specific custom transformer that identifies Update/Insert/Delete by comparing geometry and attributes to operational tables:  Terminal  Bridge  Bollard  Fairway  Berth  Node  Lock  Lock Chamber
    70. 70. Using Parameters  Published –  Passed by Django application  Format-specific (3-GML, 9-shape) and user credentials, verify authority  Private –  PostGIS connections  Schema parameters  Output location
    71. 71. Database Updates Example update database process (lock chamber)
    72. 72. Benefits  Central storage  Divided management  Detailed validation feedback  Summary table  Geographic files (downloads)  Full update history  Easy to expand (data formats, validation rules…)  Fast data update / Up-to-date database
    73. 73. Thank You!  Questions?  For more information:  