SlideShare a Scribd company logo
1 of 47
Download to read offline
CHOOSING THE
DATA CHALLENGE
Team of Experts
Dale Lutz
Co-Founder and VP of
Software Development
Iris Gutowski
Web Developer, Web
GUI Team Lead
Mark Ireland
FME Evangelist
Team of Experts (on Q&A)
Trent Kading
FME Technology Expert
Nathan Hildebrand
FME Technology Expert
Debbi Leung
FME Desktop
Technology Expert
Stephanie Warner
Webinar Expert
Topics
Covered
● Resources on FME
Transformers
● Top 10+ Transformers - Expert
recommended!
● Data specific Transformers
and FME user submitted
scenarios
● Live Q+A
in today’s webinar
#SafeWebinar
Resources Quick Add in FME Workbench
Transformer Gallery:
safe.com/transformers
Transformer Guide Booklet
fme.ly/TransformerGuide18
Great article for merging
transformers:
fme.ly/mergingtransformers
The best places to learn
about FME’s 400+ transformers
New to FME?
On the best place to learn FME:
Our Free training courses on our virtual machines
On transformer essentials:
Review the top 30 transformers listed here: fme.ly/top30
On dealing with a specific type of data?
Try typing that on the Workbench canvas to search
(e.g. PointCloud, Raster)
Or maybe you just want to refresh the basics.
Here’s what our experts have to say:
ARE YOU UP FOR A
CHALLENGE?
Calling all users in the FME Knowledge Center
Sign up for free at knowledge.safe.com
User Scenarios Uploaded
Movie Magik
1) Grab list of video files from a webpage using many html extractors and list expanders.
2) Compare that list to list of files in a folder
3) Connect to an FTP Server and grab the missing files
4) Connect to YouTube’s API and post the new video to youtube
XML eXtravagance
“Read online XML - Create as points, remove irrelevant fields etc, export to ESRI
geodatabase feature class, or if possible to several Oracle Databases representing our
different environments using in ArcGIS Portal. Add metadata to the feature class, updating
an XML file which already contains the metadata for the layer in INSPIRE format, use a
transformer to update a field in metadata with today’s date….”
Do you know which transformers to use?
Go to knowledge.safe.com to share!
1
2
Rad Radar
“I would like to be able to analyze
raster images of historical radar
from NOAA.
I would compare the value in the
radar image (a geotiff) with a linear
feature (electric line).
The radar images are stored in 5
minute intervals so I would like to
know how often the linear feature
intersected with a certain level of
radar activity.”
User Submitted Scenario #1
Rad Radar
User Submitted Scenario #1
Key transformers used to
solve the problem
NumericRasterizer:
Create a raster from vector
input
Use for: Calculating statistics
based on a designated
attribute or set of attributes of
the incoming features.
Alternative: Convert Raster to
vector and determining
intersections in Raster
Dmitri says:
Always stay in
raster as long
as possible.
Dynamic
Duo
“I have been able to create a
workspace which can achieve the
above, but it is not dynamic. If a
new table gets created in the
database then the workspace has
to be edited to include it. New
tables are commonly created so
there needs to be a workspace
solution which does not reference
specific database tables, but the
answer is beyond me!”
User Submitted Scenario #2
Dynamic Duo
User Submitted Scenario #2
Key transformer used to solve the problem
#SafeWebinar
Anonymous
Automation
“So the scenario is:
1) how do we read as many as 13
different csv files (all with the
same file name)
2) decide which feature type we
are dealing with in each csv file
3) write those records for each csv
file to the appropriate set of fields
in a MS SQL database”
User Submitted Scenario #3
Anonymous
Automation
User Submitted Scenario #3
Key transformers used to
solve the problem
Filters features by test conditions to
one or more output ports.
Removes all attributes and list
attributes, except the specific ones
you specify to be retained.
PDF Palooza
“Objective: to produce a multi-page
PDF – one page for each photo -
each page containing 4 objects
(plus Title and signature)
1) A photo
2) Large scale location plan 1:5000
3) Small scale location plan 1:100,000
4) Relevant row from points shape
file attribute table giving details of
featured photo.”
User Submitted Scenario #4
#SafeWebinar
PDF
Palooza
User Submitted
Scenario #4
Key transformers used
to solve the problem.
Use to:
Prepare features for output to PDF
Use when:
You want to detect a geometric
relationship containment,
intersection, etc between 2 features
Also Consider:
HTML ReportGenerator
to make dynamic web
reports
SCENARIOS
SOLVED?
Check in with the FME Knowledge Center
Sign up for free at knowledge.safe.com
Winds of
Change
“I need to convert this to two new
raster bands U and V. I.e. a vector
equivalent to the input bands.
But how would I apply this
calculation to NetCDF files?
Next I need to convert this raster file
to SQL Inserts that can be used to
load data into a database.”
User Submitted Scenario #9
#SafeWebinar
Winds of
Change
User Submitted Scenario #9
Key transformers used to
solve the problem
Use to calculate a new
raster from values from
existing rasters
Use to convert raster cell
values to individual
points
Use to turn a points x,y
values into attributes
LAS Vegas “I have about 4000 LAS files and I
need to generate DEM’s and
Hillshades from them. So far, I can
only process them one at a time. I’d
like to see how I can automate this
process through FME to process all
4k tiles, using the tile name as an
output name.”
User Submitted Scenario #12
#SafeWebinar
LAS Vegas
User Submitted Scenario #12
Key transformers used to
solve the problem
Draws input point, line and
polygon features onto a
numeric raster filled with
the background value.
Generates a shaded relief
effect, useful for visualizing
terrain.
Constructs a Delaunay
triangulation based on
input points and
breaklines.
Mosaics multiple raster
features into a single
raster feature.
Path
Production
“I am trying to take an area feature
class representing trails and
convert it to a line feature class.
The CenterLineReplacer has given
me some results, but the data
needs a lot of clean up and this is
where I am struggling.”
User Submitted Scenario #8
#SafeWebinar
Path
Production
User Submitted Scenario #8
Key transformers used to
solve the problem
Use for:
Replacing an area feature with its medial axis or
a straight skeleton. This transformer works best
with long, narrow areas.
Use for:
Transforming or measuring geometry features
based on a specified algorithm.
List Limbo
aka
Overlapping
Overlay
“A. I have 8 different polygon
datasets with flood model meshes
(which basically are triangles with a
depth value)
B. I have another 5 polygon datasets
with land use, which are not
complementary, i.e. they do overlap
in places
The question to be answered is: how
much land (number and area) is
affected by flooding ?
...which transformer should I use?”
User Submitted Scenario #14
List Limbo
User Submitted Scenario #14
Key transformers used to
solve the problem
Alters multiple attributes through
adding, renaming, copying,
deleting and re-ordering.
Dissolves area features by
removing common boundaries to
create larger areas.
AreaOnAreaOverlayer: Performs an
area-on-area overlay (intersection of
polygons) so that all input areas are
intersected against each other and
resultant area features are created
and output
Use Tester when:
- You want to make a different
decision based on an Attributes Value
- To split the logic flow into 2 cases IE
Test Pass Test Fail
CAD
Conversion
The requested workspace would convert a
folder of Bentley MicroStation DGNs to ESRI
Geodatabases with the following parameters:
1. Create one GDB per DGN with the new GDB
having the same name as the original DGN
2. The DGNs have a local coordinate system
but have an associated world file in the folder
3. Convert annotation, points, polygons, and
polylines while honoring the original levels in
the DGN without attaching the annotation to
the points, polygons, and polylines. (I need
Annotation as a separate Feature Class)
4. The Annotation must honor the sizing in an
MXD as it was in the original DGN
User Submitted Scenario #11
#SafeWebinar
CAD
Conversion
User Submitted Scenario #11
Key transformers used to
solve the problem
Routes a feature based on its
geometry type.
Adds one or more attributes to the
feature and optionally assigns a value
derived from constants, attribute
values, and expressions.
Common theme in
Dealing with large
data volumes
user submitted scenarios:
Video on Bulk Data
Processing: fme.ly/bulk-data
Dynamic Workflow:
fme.ly/dynamic-workflow
Q&A
Thank you!
If we didn’t get to your scenario, please post it to
knowledge.safe.com or tweet us @SafeSoftware
Transformer
Reference
Guide
Thank you! Get our free
fme.ly/TransformerGuide18
Bonus Notes
(Not included in webinar)
Tester
Use tester when:
>You want to make a different decision based on an Attributes Value
> to split the logic flow into 2 cases IE Test Pass Test Fail
Also Consider: TestFilter
> If you want to consider branching more than 2 ways consider a TestFilter
AttributeCreator
Use when you want to:
Create attributes based on the value of other attributes
Use over AttributeManager if you want simplicity and less options
Also Consider: AttributeManager
Pro Secret: AttributeCreator and Manager are the same under the covers
FeatureMerger
Use when you want to bring data together
Allows for a basic join on matching attribute values
Can take a combination of ports to produce inner, left right and full outer joins
Offers complex ways to assemble geometry as either aggregates or reassembling
Use when you need an all - purpose tool
Also Consider FeatureJoiner for performance
SQL Caller/DBJoiner- Merging in the DB often faster can use SQL
InlineQuerier if you know sql for more flexibility can use SQL on non -dbs
AttributeManager
Use when you need to do anything attribute
Create, delete, rename and copy
Do all your attribute manipulation in 1 place!
Keep your workspaces tidy
Reprojector
Use when you want to convert a geometries coordinates to a new coord system
you want the actual coord values to change (actual data transformation)
Don’t confuse with CoordinateSystemSetter when you have the right coordinates
but just need to define a new system
Also consider ESriReprojector CSMapReprojector which use different engines
Creator
Use when you need a “dummy” feature to trigger something
Want to create a simple test geometry
Creator ->Http Caller> …
Can bring in a wkt geometry easily
Also consider RasterCreator, PointCloudCreator
AttributeRenamer
Does a basic rename
Consider combining with other tasks in the AttributeManager
TestFilter
Use when you need to split out features into multiple 2+ paths depending on
attribute values
IE branch a table out to different writers
AttributeKeeper
Use when you need to clean up
Cut your schema down to the set of things you want
Can improve performance and reduce memory overhead
Also Consider AttributeRemover to specify what to get rid of instead of what to
keep
Clipper
Use when you want to clip a geometry to a predefined area
Also consider AreaOnAreaOverlayer
AttributeValidator
Expose hidden attributes
Logger
Can be used to troubleshoot tricky data type problems
Transformer
Reference
Guide
Thank you! Get our free
fme.ly/TransformerGuide18

More Related Content

What's hot

DMM9 - Data Migration Testing
DMM9 - Data Migration TestingDMM9 - Data Migration Testing
DMM9 - Data Migration Testing
Nick van Beest
 

What's hot (15)

Hci solution with VxRail
Hci solution with VxRailHci solution with VxRail
Hci solution with VxRail
 
Project kick off
Project kick offProject kick off
Project kick off
 
Management of Risk and its integration within ITIL
Management of Risk and its integration within ITILManagement of Risk and its integration within ITIL
Management of Risk and its integration within ITIL
 
The Top Ten things that have been proven to effect software reliability
The Top Ten things that have been proven to effect software reliabilityThe Top Ten things that have been proven to effect software reliability
The Top Ten things that have been proven to effect software reliability
 
OpenText Archive Server on Azure
OpenText Archive Server on AzureOpenText Archive Server on Azure
OpenText Archive Server on Azure
 
Five common customer use cases for Virtual SAN - VMworld US / 2015
Five common customer use cases for Virtual SAN - VMworld US / 2015Five common customer use cases for Virtual SAN - VMworld US / 2015
Five common customer use cases for Virtual SAN - VMworld US / 2015
 
Object Store V2 Workshop
Object Store V2 WorkshopObject Store V2 Workshop
Object Store V2 Workshop
 
VMware vSphere
VMware vSphereVMware vSphere
VMware vSphere
 
HPE SimpliVity install mgmt guide 201907-01 (Taiwan-Chinese) ;HPE SimpliVity ...
HPE SimpliVity install mgmt guide 201907-01 (Taiwan-Chinese) ;HPE SimpliVity ...HPE SimpliVity install mgmt guide 201907-01 (Taiwan-Chinese) ;HPE SimpliVity ...
HPE SimpliVity install mgmt guide 201907-01 (Taiwan-Chinese) ;HPE SimpliVity ...
 
03 workflow
03 workflow03 workflow
03 workflow
 
2015 04 Preparing for the SAP S/4HANA Migration
2015 04 Preparing for the SAP S/4HANA Migration2015 04 Preparing for the SAP S/4HANA Migration
2015 04 Preparing for the SAP S/4HANA Migration
 
IBM MQ Update, including 9.1.2 CD
IBM MQ Update, including 9.1.2 CDIBM MQ Update, including 9.1.2 CD
IBM MQ Update, including 9.1.2 CD
 
Domain driven design
Domain driven designDomain driven design
Domain driven design
 
DMM9 - Data Migration Testing
DMM9 - Data Migration TestingDMM9 - Data Migration Testing
DMM9 - Data Migration Testing
 
vRealize Operation 7.5 What's new
vRealize Operation 7.5 What's newvRealize Operation 7.5 What's new
vRealize Operation 7.5 What's new
 

Similar to Choosing the Right Transformer for Your Data Challenge

advancedzplmacroprogramming_081820.pptx
advancedzplmacroprogramming_081820.pptxadvancedzplmacroprogramming_081820.pptx
advancedzplmacroprogramming_081820.pptx
ssuser6a1dbf
 
22827361 ab initio-fa-qs
22827361 ab initio-fa-qs22827361 ab initio-fa-qs
22827361 ab initio-fa-qs
Capgemini
 
Determan SummerSim_submit_rev3
Determan SummerSim_submit_rev3Determan SummerSim_submit_rev3
Determan SummerSim_submit_rev3
John Determan
 
Google Cloud Computing on Google Developer 2008 Day
Google Cloud Computing on Google Developer 2008 DayGoogle Cloud Computing on Google Developer 2008 Day
Google Cloud Computing on Google Developer 2008 Day
programmermag
 

Similar to Choosing the Right Transformer for Your Data Challenge (20)

How to Master Raster (Faster) - Tips and Examples
How to Master Raster (Faster) - Tips and ExamplesHow to Master Raster (Faster) - Tips and Examples
How to Master Raster (Faster) - Tips and Examples
 
Lifting the Hood of FME Engine 2022.0
Lifting the Hood of FME Engine 2022.0Lifting the Hood of FME Engine 2022.0
Lifting the Hood of FME Engine 2022.0
 
Advanced web application architecture - Talk
Advanced web application architecture - TalkAdvanced web application architecture - Talk
Advanced web application architecture - Talk
 
advancedzplmacroprogramming_081820.pptx
advancedzplmacroprogramming_081820.pptxadvancedzplmacroprogramming_081820.pptx
advancedzplmacroprogramming_081820.pptx
 
Deep learning and streaming in Apache Spark 2.2 by Matei Zaharia
Deep learning and streaming in Apache Spark 2.2 by Matei ZahariaDeep learning and streaming in Apache Spark 2.2 by Matei Zaharia
Deep learning and streaming in Apache Spark 2.2 by Matei Zaharia
 
22827361 ab initio-fa-qs
22827361 ab initio-fa-qs22827361 ab initio-fa-qs
22827361 ab initio-fa-qs
 
SDWest2005Goetsch
SDWest2005GoetschSDWest2005Goetsch
SDWest2005Goetsch
 
Designing a Scalable Twitter - Patterns for Designing Scalable Real-Time Web ...
Designing a Scalable Twitter - Patterns for Designing Scalable Real-Time Web ...Designing a Scalable Twitter - Patterns for Designing Scalable Real-Time Web ...
Designing a Scalable Twitter - Patterns for Designing Scalable Real-Time Web ...
 
Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist
 
Questions Log: Dynamic Cubes – Set to Retire Transformer?
Questions Log: Dynamic Cubes – Set to Retire Transformer?Questions Log: Dynamic Cubes – Set to Retire Transformer?
Questions Log: Dynamic Cubes – Set to Retire Transformer?
 
Determan SummerSim_submit_rev3
Determan SummerSim_submit_rev3Determan SummerSim_submit_rev3
Determan SummerSim_submit_rev3
 
GraphConnect 2014 SF: Neo4j at Scale using Enterprise Integration Patterns
GraphConnect 2014 SF: Neo4j at Scale using Enterprise Integration PatternsGraphConnect 2014 SF: Neo4j at Scale using Enterprise Integration Patterns
GraphConnect 2014 SF: Neo4j at Scale using Enterprise Integration Patterns
 
Productionalizing ML : Real Experience
Productionalizing ML : Real ExperienceProductionalizing ML : Real Experience
Productionalizing ML : Real Experience
 
Google Cloud Computing on Google Developer 2008 Day
Google Cloud Computing on Google Developer 2008 DayGoogle Cloud Computing on Google Developer 2008 Day
Google Cloud Computing on Google Developer 2008 Day
 
Web engineering
Web engineeringWeb engineering
Web engineering
 
Multi-tenancy with Rails
Multi-tenancy with RailsMulti-tenancy with Rails
Multi-tenancy with Rails
 
Achieve big data analytic platform with lambda architecture on cloud
Achieve big data analytic platform with lambda architecture on cloudAchieve big data analytic platform with lambda architecture on cloud
Achieve big data analytic platform with lambda architecture on cloud
 
Application Timeline Server - Past, Present and Future
Application Timeline Server - Past, Present and FutureApplication Timeline Server - Past, Present and Future
Application Timeline Server - Past, Present and Future
 
Application Timeline Server - Past, Present and Future
Application Timeline Server - Past, Present and FutureApplication Timeline Server - Past, Present and Future
Application Timeline Server - Past, Present and Future
 
Recommender Systems from A to Z – Real-Time Deployment
Recommender Systems from A to Z – Real-Time DeploymentRecommender Systems from A to Z – Real-Time Deployment
Recommender Systems from A to Z – Real-Time Deployment
 

More from Safe Software

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
Safe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 

More from Safe Software (20)

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 

Recently uploaded

+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
Health
 
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
chiefasafspells
 
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Medical / Health Care (+971588192166) Mifepristone and Misoprostol tablets 200mg
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
masabamasaba
 
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
masabamasaba
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
masabamasaba
 
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
masabamasaba
 

Recently uploaded (20)

%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
 
WSO2CON 2024 - API Management Usage at La Poste and Its Impact on Business an...
WSO2CON 2024 - API Management Usage at La Poste and Its Impact on Business an...WSO2CON 2024 - API Management Usage at La Poste and Its Impact on Business an...
WSO2CON 2024 - API Management Usage at La Poste and Its Impact on Business an...
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
 
AI & Machine Learning Presentation Template
AI & Machine Learning Presentation TemplateAI & Machine Learning Presentation Template
AI & Machine Learning Presentation Template
 
%in Rustenburg+277-882-255-28 abortion pills for sale in Rustenburg
%in Rustenburg+277-882-255-28 abortion pills for sale in Rustenburg%in Rustenburg+277-882-255-28 abortion pills for sale in Rustenburg
%in Rustenburg+277-882-255-28 abortion pills for sale in Rustenburg
 
Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the pastArchitecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
 
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareAnnouncing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
 
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
 
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
 
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
 
WSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaSWSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaS
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
 
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go PlatformlessWSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go Platformless
 
Artyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptxArtyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptx
 
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
 
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
 
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
 
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
 
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
 

Choosing the Right Transformer for Your Data Challenge

  • 2. Team of Experts Dale Lutz Co-Founder and VP of Software Development Iris Gutowski Web Developer, Web GUI Team Lead Mark Ireland FME Evangelist
  • 3. Team of Experts (on Q&A) Trent Kading FME Technology Expert Nathan Hildebrand FME Technology Expert Debbi Leung FME Desktop Technology Expert Stephanie Warner Webinar Expert
  • 4. Topics Covered ● Resources on FME Transformers ● Top 10+ Transformers - Expert recommended! ● Data specific Transformers and FME user submitted scenarios ● Live Q+A in today’s webinar #SafeWebinar
  • 5. Resources Quick Add in FME Workbench Transformer Gallery: safe.com/transformers Transformer Guide Booklet fme.ly/TransformerGuide18 Great article for merging transformers: fme.ly/mergingtransformers The best places to learn about FME’s 400+ transformers
  • 6. New to FME? On the best place to learn FME: Our Free training courses on our virtual machines On transformer essentials: Review the top 30 transformers listed here: fme.ly/top30 On dealing with a specific type of data? Try typing that on the Workbench canvas to search (e.g. PointCloud, Raster) Or maybe you just want to refresh the basics. Here’s what our experts have to say:
  • 7. ARE YOU UP FOR A CHALLENGE? Calling all users in the FME Knowledge Center Sign up for free at knowledge.safe.com
  • 8. User Scenarios Uploaded Movie Magik 1) Grab list of video files from a webpage using many html extractors and list expanders. 2) Compare that list to list of files in a folder 3) Connect to an FTP Server and grab the missing files 4) Connect to YouTube’s API and post the new video to youtube XML eXtravagance “Read online XML - Create as points, remove irrelevant fields etc, export to ESRI geodatabase feature class, or if possible to several Oracle Databases representing our different environments using in ArcGIS Portal. Add metadata to the feature class, updating an XML file which already contains the metadata for the layer in INSPIRE format, use a transformer to update a field in metadata with today’s date….” Do you know which transformers to use? Go to knowledge.safe.com to share! 1 2
  • 9. Rad Radar “I would like to be able to analyze raster images of historical radar from NOAA. I would compare the value in the radar image (a geotiff) with a linear feature (electric line). The radar images are stored in 5 minute intervals so I would like to know how often the linear feature intersected with a certain level of radar activity.” User Submitted Scenario #1
  • 10. Rad Radar User Submitted Scenario #1 Key transformers used to solve the problem NumericRasterizer: Create a raster from vector input Use for: Calculating statistics based on a designated attribute or set of attributes of the incoming features. Alternative: Convert Raster to vector and determining intersections in Raster Dmitri says: Always stay in raster as long as possible.
  • 11.
  • 12. Dynamic Duo “I have been able to create a workspace which can achieve the above, but it is not dynamic. If a new table gets created in the database then the workspace has to be edited to include it. New tables are commonly created so there needs to be a workspace solution which does not reference specific database tables, but the answer is beyond me!” User Submitted Scenario #2
  • 13. Dynamic Duo User Submitted Scenario #2 Key transformer used to solve the problem #SafeWebinar
  • 14. Anonymous Automation “So the scenario is: 1) how do we read as many as 13 different csv files (all with the same file name) 2) decide which feature type we are dealing with in each csv file 3) write those records for each csv file to the appropriate set of fields in a MS SQL database” User Submitted Scenario #3
  • 15. Anonymous Automation User Submitted Scenario #3 Key transformers used to solve the problem Filters features by test conditions to one or more output ports. Removes all attributes and list attributes, except the specific ones you specify to be retained.
  • 16. PDF Palooza “Objective: to produce a multi-page PDF – one page for each photo - each page containing 4 objects (plus Title and signature) 1) A photo 2) Large scale location plan 1:5000 3) Small scale location plan 1:100,000 4) Relevant row from points shape file attribute table giving details of featured photo.” User Submitted Scenario #4 #SafeWebinar
  • 17. PDF Palooza User Submitted Scenario #4 Key transformers used to solve the problem. Use to: Prepare features for output to PDF Use when: You want to detect a geometric relationship containment, intersection, etc between 2 features Also Consider: HTML ReportGenerator to make dynamic web reports
  • 18.
  • 19. SCENARIOS SOLVED? Check in with the FME Knowledge Center Sign up for free at knowledge.safe.com
  • 20. Winds of Change “I need to convert this to two new raster bands U and V. I.e. a vector equivalent to the input bands. But how would I apply this calculation to NetCDF files? Next I need to convert this raster file to SQL Inserts that can be used to load data into a database.” User Submitted Scenario #9 #SafeWebinar
  • 21. Winds of Change User Submitted Scenario #9 Key transformers used to solve the problem Use to calculate a new raster from values from existing rasters Use to convert raster cell values to individual points Use to turn a points x,y values into attributes
  • 22.
  • 23. LAS Vegas “I have about 4000 LAS files and I need to generate DEM’s and Hillshades from them. So far, I can only process them one at a time. I’d like to see how I can automate this process through FME to process all 4k tiles, using the tile name as an output name.” User Submitted Scenario #12 #SafeWebinar
  • 24. LAS Vegas User Submitted Scenario #12 Key transformers used to solve the problem Draws input point, line and polygon features onto a numeric raster filled with the background value. Generates a shaded relief effect, useful for visualizing terrain. Constructs a Delaunay triangulation based on input points and breaklines. Mosaics multiple raster features into a single raster feature.
  • 25. Path Production “I am trying to take an area feature class representing trails and convert it to a line feature class. The CenterLineReplacer has given me some results, but the data needs a lot of clean up and this is where I am struggling.” User Submitted Scenario #8 #SafeWebinar
  • 26. Path Production User Submitted Scenario #8 Key transformers used to solve the problem Use for: Replacing an area feature with its medial axis or a straight skeleton. This transformer works best with long, narrow areas. Use for: Transforming or measuring geometry features based on a specified algorithm.
  • 27. List Limbo aka Overlapping Overlay “A. I have 8 different polygon datasets with flood model meshes (which basically are triangles with a depth value) B. I have another 5 polygon datasets with land use, which are not complementary, i.e. they do overlap in places The question to be answered is: how much land (number and area) is affected by flooding ? ...which transformer should I use?” User Submitted Scenario #14
  • 28. List Limbo User Submitted Scenario #14 Key transformers used to solve the problem Alters multiple attributes through adding, renaming, copying, deleting and re-ordering. Dissolves area features by removing common boundaries to create larger areas. AreaOnAreaOverlayer: Performs an area-on-area overlay (intersection of polygons) so that all input areas are intersected against each other and resultant area features are created and output Use Tester when: - You want to make a different decision based on an Attributes Value - To split the logic flow into 2 cases IE Test Pass Test Fail
  • 29. CAD Conversion The requested workspace would convert a folder of Bentley MicroStation DGNs to ESRI Geodatabases with the following parameters: 1. Create one GDB per DGN with the new GDB having the same name as the original DGN 2. The DGNs have a local coordinate system but have an associated world file in the folder 3. Convert annotation, points, polygons, and polylines while honoring the original levels in the DGN without attaching the annotation to the points, polygons, and polylines. (I need Annotation as a separate Feature Class) 4. The Annotation must honor the sizing in an MXD as it was in the original DGN User Submitted Scenario #11 #SafeWebinar
  • 30. CAD Conversion User Submitted Scenario #11 Key transformers used to solve the problem Routes a feature based on its geometry type. Adds one or more attributes to the feature and optionally assigns a value derived from constants, attribute values, and expressions.
  • 31. Common theme in Dealing with large data volumes user submitted scenarios: Video on Bulk Data Processing: fme.ly/bulk-data Dynamic Workflow: fme.ly/dynamic-workflow
  • 32. Q&A Thank you! If we didn’t get to your scenario, please post it to knowledge.safe.com or tweet us @SafeSoftware
  • 33. Transformer Reference Guide Thank you! Get our free fme.ly/TransformerGuide18
  • 35. Tester Use tester when: >You want to make a different decision based on an Attributes Value > to split the logic flow into 2 cases IE Test Pass Test Fail Also Consider: TestFilter > If you want to consider branching more than 2 ways consider a TestFilter
  • 36. AttributeCreator Use when you want to: Create attributes based on the value of other attributes Use over AttributeManager if you want simplicity and less options Also Consider: AttributeManager Pro Secret: AttributeCreator and Manager are the same under the covers
  • 37. FeatureMerger Use when you want to bring data together Allows for a basic join on matching attribute values Can take a combination of ports to produce inner, left right and full outer joins Offers complex ways to assemble geometry as either aggregates or reassembling Use when you need an all - purpose tool Also Consider FeatureJoiner for performance SQL Caller/DBJoiner- Merging in the DB often faster can use SQL InlineQuerier if you know sql for more flexibility can use SQL on non -dbs
  • 38. AttributeManager Use when you need to do anything attribute Create, delete, rename and copy Do all your attribute manipulation in 1 place! Keep your workspaces tidy
  • 39. Reprojector Use when you want to convert a geometries coordinates to a new coord system you want the actual coord values to change (actual data transformation) Don’t confuse with CoordinateSystemSetter when you have the right coordinates but just need to define a new system Also consider ESriReprojector CSMapReprojector which use different engines
  • 40. Creator Use when you need a “dummy” feature to trigger something Want to create a simple test geometry Creator ->Http Caller> … Can bring in a wkt geometry easily Also consider RasterCreator, PointCloudCreator
  • 41. AttributeRenamer Does a basic rename Consider combining with other tasks in the AttributeManager
  • 42. TestFilter Use when you need to split out features into multiple 2+ paths depending on attribute values IE branch a table out to different writers
  • 43. AttributeKeeper Use when you need to clean up Cut your schema down to the set of things you want Can improve performance and reduce memory overhead Also Consider AttributeRemover to specify what to get rid of instead of what to keep
  • 44. Clipper Use when you want to clip a geometry to a predefined area Also consider AreaOnAreaOverlayer
  • 46. Logger Can be used to troubleshoot tricky data type problems
  • 47. Transformer Reference Guide Thank you! Get our free fme.ly/TransformerGuide18