SlideShare a Scribd company logo
1 of 29
Download to read offline
Donʼt Bug Out!
The Ins and Outs of
Debugging FME Workflows
Meet the Presenters
Laura Wu
Director, Customer Support
Mark Stoakes
Senior Manager, FME Application Experts
Agenda
● When do things go off-the-rails?
● What does that look like?
● Tools for Debugging Workspaces
● Testing and Debugging in FME Server
● Itʼs a wrap
Ever developed a workflow, but
reached a roadblock of an
unexpected result or error message
you werenʼt sure how to resolve?
Type into the chat panel:
What’s your preferred tool for
troubleshooting your workspace?
Keywords Defined
Testing is the process of finding where errors occur in your workflow
Troubleshooting tracking down root-cause of issues
Debugging is the process to correct the errors found and isolated during testing and
troubleshooting
Go Off-The-Rails: to lose control and start to behave in a way that is not normal or
acceptable
When are things likely to go off-the-rails
● During workbench authoring
○ Testing out how transformers work
○ Experimenting with different approaches - should I use PointOnAreaOverlayer
or SpatialFilter or SpatialRelator?
● Bad data - oops! I thought all my data was perfect and consistent.
● When the data changes - different data brings different problems
○ For example you switch from your test data to your full dataset
● Differences between FME Desktop and FME Server
● FME errors / crashes - which never ever happens!
The symptoms?
● FME log:
○ ERROR
○ WARN
○ INFO
● Unexpected or incorrect results
● Rejected data
● No results
Error running translation.
Get Comfortable in FME Workbench
Tune FME Workbench:
● Modify your Toolbar
● Workbench window
layout - maximize your real
Estate
● Data Inspector vs. Visual Preview?
Use both…
Example
Tools for Troubleshooting in Workbench
● Feature Caching and Visual Preview 🔗
○ Collapse bookmarks for large workspaces
○ Caching doesnʼt always give the results you expect
○ Feature caching is not so good for very large datasets (and rasters)
● Data Inspector (and Visual Preview)
○ Graphics View
○ Table View
○ Feature Information
● Break Points 🔗
● Disable objects
● Workbench Search
● Transformers
○ fme_rejection_code
Data Inspection reminders…
● Data Inspector (and Visual Preview 🔗)
○ Graphics View
■ Background Maps
○ Table View
■ Columns / Search
○ Feature Information
■ Additional Attributes displayed
○ Display Control
More Tools for Troubleshooting in Workbench
● Logging - old school but effective
○ Reviewing your log file
○ Where is my log file?
<working folder><myworkspace>.log
○ Links to transformers
● Spatial Log File
○ <myworkspace>_log.ffs
More Tools for Troubleshooting in Workbench
● Logging - old school but effective
○ Reviewing your log file
○ Where is my log file?
<working folder><myworkspace>.log
○ Links to transformers
● Spatial Log File
○ <myworkspace>_log.ffs
More Tools for Troubleshooting in Workbench
● Logging - old school but effective
○ Reviewing your log file
○ Where is my log file?
<working folder><myworkspace>.log
○ Links to transformers
● Spatial Log File
○ <myworkspace>_log.ffs
● Debug Logging
○ FME Options > Translation > Log Message Filter > Log Debug
○ Extra detail for complex errors
○ Good for debugging with the HTTPCaller, SQLExecutor, SQLCreator
Simply your data: Reduce the volume of data
● Reduces run time
● Easier to find issues
Handy Transformers and Parameters for Simplifying Data
● TestFilter
● Sampler
● Max. Features to Read /
Feature Types to Read
Simplify your data when authoring
Where oh where did I make that change?
Workbench Search 🔗
Workspace Compare 🔗
● Compare the differences
between two workspaces
● Merge the changes
Isolating the issue
Make a Small Example
● Save as Template
○ Save with Cache
● Recorder / Player
● Sampler
Poll:
Have you seen any new troubleshooting
ideas that you might try?
Debugging Scripts
Python scripts can appear in various places in Workbench
● PythonCaller
● Startup/Shutdown Python Script
● Scripted Parameters
Script troubleshooting approaches:
● Python print statements
● Logger transformer
● Build complex scripts in a separate Python editor and import into FME
Many of the tips we have reviewed already also apply to FME Server
Debugging on FME Server
If a workspace doesnʼt run correctly on FME Server, try it on Desktop first
If it works on Desktop, but not FME Server, look for differences in the environments
● Permissions
● Dependencies
○ File paths
○ Missing transformers
○ Missing Python modules
○ Missing Web and Database Connections
○ Mismatched FME Versions
● Nondescript errors
○ Look at fmeprocessmonitorengine.log
Example
Even more thoughts on this topic…
Performance troubleshooting 🔗
Database Errors 🔗
● Errors can be buried in a transaction
○ The entire transaction is rolled back
○ Try Transaction Interval = 1
● Save As Workspace Template doesnʼt encompass database data
○ Recorder / Player
● Test SQL scripts in the appropriate database manager
● Database system log files
Some Best Practices
● Versioning workspaces
○ Publishing to FME Server & Commit
○ Workspace Compare
● Good workspace practices help with maintenance 🔗 🎓
○ Annotations / bookmarks
○ Look for repetition in your workspace - multiple readers / writers. Many
Testers
○ Should you rename Transformers
Summary
So weʼve covered:
● What it looks like when you do
encounter issues
● Tools for troubleshooting
workspaces
● Testing and debugging in FME
Server
Resources
● Feature Caching🔗
● Break Points🔗
● Visual Preview 🔗
● Workspace Compare 🔗
● Workbench Best Practices 🔗
● Debugging Workspaces on
FME Server 🔗
Claim Your Community Badge
Get community badges for
watching webinars!
fme.ly/WebinarBadge Todayʼs Code: PSLGW
Q&A
Thank you!
Free Trial | Upgrade
Chat with us info@safe.com
Connect with us in the Community
Connect with us for more FME
Please share
your feedback
with us through
the webinar
survey!

More Related Content

Similar to Don't Bug Out! Debugging FME Workflows

What Your Tech Lead Thinks You Know (But Didn't Teach You)
What Your Tech Lead Thinks You Know (But Didn't Teach You)What Your Tech Lead Thinks You Know (But Didn't Teach You)
What Your Tech Lead Thinks You Know (But Didn't Teach You)Chris Riccomini
 
Destination Documentation: How Not to Get Lost in Your Org
Destination Documentation: How Not to Get Lost in Your OrgDestination Documentation: How Not to Get Lost in Your Org
Destination Documentation: How Not to Get Lost in Your Orgcsupilowski
 
How to establish ways of working that allows shifting-left of the automation ...
How to establish ways of working that allows shifting-left of the automation ...How to establish ways of working that allows shifting-left of the automation ...
How to establish ways of working that allows shifting-left of the automation ...Max Barrass
 
Keeping business logic out of your UIs
Keeping business logic out of your UIsKeeping business logic out of your UIs
Keeping business logic out of your UIsPetter Holmström
 
Getting Started with Data Integration: FME Desktop
Getting Started with Data Integration: FME DesktopGetting Started with Data Integration: FME Desktop
Getting Started with Data Integration: FME DesktopSafe Software
 
Talkin bout Flow - Meighan Brodkey WIT Devs
Talkin bout Flow - Meighan Brodkey WIT Devs Talkin bout Flow - Meighan Brodkey WIT Devs
Talkin bout Flow - Meighan Brodkey WIT Devs Meighan Brodkey
 
May 2021 Spark Testing ... or how to farm reputation on StackOverflow
May 2021 Spark Testing ... or how to farm reputation on StackOverflowMay 2021 Spark Testing ... or how to farm reputation on StackOverflow
May 2021 Spark Testing ... or how to farm reputation on StackOverflowAdam Doyle
 
Log4j2 - A deep dive into the logging services in Mulesoft with On-Prem deplo...
Log4j2 - A deep dive into the logging services in Mulesoft with On-Prem deplo...Log4j2 - A deep dive into the logging services in Mulesoft with On-Prem deplo...
Log4j2 - A deep dive into the logging services in Mulesoft with On-Prem deplo...MysoreMuleSoftMeetup
 
How to create a useful MySQL bug report
How to create a useful MySQL bug reportHow to create a useful MySQL bug report
How to create a useful MySQL bug reportValerii Kravchuk
 
Validating Big Data Pipelines - Big Data Spain 2018
Validating Big Data Pipelines - Big Data Spain 2018Validating Big Data Pipelines - Big Data Spain 2018
Validating Big Data Pipelines - Big Data Spain 2018Holden Karau
 
Validating spark ml jobs stopping failures before production on Apache Spark ...
Validating spark ml jobs stopping failures before production on Apache Spark ...Validating spark ml jobs stopping failures before production on Apache Spark ...
Validating spark ml jobs stopping failures before production on Apache Spark ...Holden Karau
 
Improve the deployment process step by step
Improve the deployment process step by stepImprove the deployment process step by step
Improve the deployment process step by stepDaniel Fahlke
 
Présentation du FME World Tour 2018 à Québec
Présentation du FME World Tour 2018 à QuébecPrésentation du FME World Tour 2018 à Québec
Présentation du FME World Tour 2018 à QuébecGuillaume Genest
 
Validating Big Data Jobs—Stopping Failures Before Production on Apache Spark...
 Validating Big Data Jobs—Stopping Failures Before Production on Apache Spark... Validating Big Data Jobs—Stopping Failures Before Production on Apache Spark...
Validating Big Data Jobs—Stopping Failures Before Production on Apache Spark...Databricks
 
Php Inspections (EA Extended): The Cookbook
Php Inspections (EA Extended): The CookbookPhp Inspections (EA Extended): The Cookbook
Php Inspections (EA Extended): The CookbookVladimir Reznichenko
 
Python for Data Logistics
Python for Data LogisticsPython for Data Logistics
Python for Data LogisticsKen Farmer
 
Where is my bottleneck? Performance troubleshooting in Flink
Where is my bottleneck? Performance troubleshooting in FlinkWhere is my bottleneck? Performance troubleshooting in Flink
Where is my bottleneck? Performance troubleshooting in FlinkFlink Forward
 

Similar to Don't Bug Out! Debugging FME Workflows (20)

What Your Tech Lead Thinks You Know (But Didn't Teach You)
What Your Tech Lead Thinks You Know (But Didn't Teach You)What Your Tech Lead Thinks You Know (But Didn't Teach You)
What Your Tech Lead Thinks You Know (But Didn't Teach You)
 
Destination Documentation: How Not to Get Lost in Your Org
Destination Documentation: How Not to Get Lost in Your OrgDestination Documentation: How Not to Get Lost in Your Org
Destination Documentation: How Not to Get Lost in Your Org
 
How to establish ways of working that allows shifting-left of the automation ...
How to establish ways of working that allows shifting-left of the automation ...How to establish ways of working that allows shifting-left of the automation ...
How to establish ways of working that allows shifting-left of the automation ...
 
Open Day July 2019
Open Day July 2019Open Day July 2019
Open Day July 2019
 
Keeping business logic out of your UIs
Keeping business logic out of your UIsKeeping business logic out of your UIs
Keeping business logic out of your UIs
 
Getting Started with Data Integration: FME Desktop
Getting Started with Data Integration: FME DesktopGetting Started with Data Integration: FME Desktop
Getting Started with Data Integration: FME Desktop
 
Lug
LugLug
Lug
 
Talkin bout Flow - Meighan Brodkey WIT Devs
Talkin bout Flow - Meighan Brodkey WIT Devs Talkin bout Flow - Meighan Brodkey WIT Devs
Talkin bout Flow - Meighan Brodkey WIT Devs
 
May 2021 Spark Testing ... or how to farm reputation on StackOverflow
May 2021 Spark Testing ... or how to farm reputation on StackOverflowMay 2021 Spark Testing ... or how to farm reputation on StackOverflow
May 2021 Spark Testing ... or how to farm reputation on StackOverflow
 
Log4j2 - A deep dive into the logging services in Mulesoft with On-Prem deplo...
Log4j2 - A deep dive into the logging services in Mulesoft with On-Prem deplo...Log4j2 - A deep dive into the logging services in Mulesoft with On-Prem deplo...
Log4j2 - A deep dive into the logging services in Mulesoft with On-Prem deplo...
 
How to create a useful MySQL bug report
How to create a useful MySQL bug reportHow to create a useful MySQL bug report
How to create a useful MySQL bug report
 
Validating Big Data Pipelines - Big Data Spain 2018
Validating Big Data Pipelines - Big Data Spain 2018Validating Big Data Pipelines - Big Data Spain 2018
Validating Big Data Pipelines - Big Data Spain 2018
 
Validating spark ml jobs stopping failures before production on Apache Spark ...
Validating spark ml jobs stopping failures before production on Apache Spark ...Validating spark ml jobs stopping failures before production on Apache Spark ...
Validating spark ml jobs stopping failures before production on Apache Spark ...
 
Improve the deployment process step by step
Improve the deployment process step by stepImprove the deployment process step by step
Improve the deployment process step by step
 
Présentation du FME World Tour 2018 à Québec
Présentation du FME World Tour 2018 à QuébecPrésentation du FME World Tour 2018 à Québec
Présentation du FME World Tour 2018 à Québec
 
Validating Big Data Jobs—Stopping Failures Before Production on Apache Spark...
 Validating Big Data Jobs—Stopping Failures Before Production on Apache Spark... Validating Big Data Jobs—Stopping Failures Before Production on Apache Spark...
Validating Big Data Jobs—Stopping Failures Before Production on Apache Spark...
 
Php Inspections (EA Extended): The Cookbook
Php Inspections (EA Extended): The CookbookPhp Inspections (EA Extended): The Cookbook
Php Inspections (EA Extended): The Cookbook
 
Python for Data Logistics
Python for Data LogisticsPython for Data Logistics
Python for Data Logistics
 
Google
GoogleGoogle
Google
 
Where is my bottleneck? Performance troubleshooting in Flink
Where is my bottleneck? Performance troubleshooting in FlinkWhere is my bottleneck? Performance troubleshooting in Flink
Where is my bottleneck? Performance troubleshooting in Flink
 

More from Safe Software

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemSafe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISSafe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersSafe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsSafe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategySafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Safe Software
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMESafe Software
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Safe Software
 
Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Safe Software
 

More from Safe Software (20)

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FME
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...
 
Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework
 

Recently uploaded

Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝soniya singh
 
What are the features of Vehicle Tracking System?
What are the features of Vehicle Tracking System?What are the features of Vehicle Tracking System?
What are the features of Vehicle Tracking System?Watsoo Telematics
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...OnePlan Solutions
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptkotipi9215
 
Engage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyEngage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyFrank van der Linden
 
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfThe Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfkalichargn70th171
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfjoe51371421
 
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationProject Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationkaushalgiri8080
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantAxelRicardoTrocheRiq
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)OPEN KNOWLEDGE GmbH
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataBradBedford3
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software DevelopersVinodh Ram
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxTier1 app
 
XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsXpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsMehedi Hasan Shohan
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...soniya singh
 

Recently uploaded (20)

Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
 
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
 
What are the features of Vehicle Tracking System?
What are the features of Vehicle Tracking System?What are the features of Vehicle Tracking System?
What are the features of Vehicle Tracking System?
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.ppt
 
Engage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyEngage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The Ugly
 
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfThe Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
 
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationProject Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanation
 
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service ConsultantSalesforce Certified Field Service Consultant
Salesforce Certified Field Service Consultant
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software Developers
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
 
XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsXpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software Solutions
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
 

Don't Bug Out! Debugging FME Workflows

  • 1. Donʼt Bug Out! The Ins and Outs of Debugging FME Workflows
  • 2. Meet the Presenters Laura Wu Director, Customer Support Mark Stoakes Senior Manager, FME Application Experts
  • 3. Agenda ● When do things go off-the-rails? ● What does that look like? ● Tools for Debugging Workspaces ● Testing and Debugging in FME Server ● Itʼs a wrap
  • 4. Ever developed a workflow, but reached a roadblock of an unexpected result or error message you werenʼt sure how to resolve?
  • 5. Type into the chat panel: What’s your preferred tool for troubleshooting your workspace?
  • 6. Keywords Defined Testing is the process of finding where errors occur in your workflow Troubleshooting tracking down root-cause of issues Debugging is the process to correct the errors found and isolated during testing and troubleshooting Go Off-The-Rails: to lose control and start to behave in a way that is not normal or acceptable
  • 7. When are things likely to go off-the-rails ● During workbench authoring ○ Testing out how transformers work ○ Experimenting with different approaches - should I use PointOnAreaOverlayer or SpatialFilter or SpatialRelator? ● Bad data - oops! I thought all my data was perfect and consistent. ● When the data changes - different data brings different problems ○ For example you switch from your test data to your full dataset ● Differences between FME Desktop and FME Server ● FME errors / crashes - which never ever happens!
  • 8. The symptoms? ● FME log: ○ ERROR ○ WARN ○ INFO ● Unexpected or incorrect results ● Rejected data ● No results Error running translation.
  • 9. Get Comfortable in FME Workbench Tune FME Workbench: ● Modify your Toolbar ● Workbench window layout - maximize your real Estate ● Data Inspector vs. Visual Preview? Use both…
  • 11. Tools for Troubleshooting in Workbench ● Feature Caching and Visual Preview 🔗 ○ Collapse bookmarks for large workspaces ○ Caching doesnʼt always give the results you expect ○ Feature caching is not so good for very large datasets (and rasters) ● Data Inspector (and Visual Preview) ○ Graphics View ○ Table View ○ Feature Information ● Break Points 🔗 ● Disable objects ● Workbench Search ● Transformers ○ fme_rejection_code
  • 12. Data Inspection reminders… ● Data Inspector (and Visual Preview 🔗) ○ Graphics View ■ Background Maps ○ Table View ■ Columns / Search ○ Feature Information ■ Additional Attributes displayed ○ Display Control
  • 13. More Tools for Troubleshooting in Workbench ● Logging - old school but effective ○ Reviewing your log file ○ Where is my log file? <working folder><myworkspace>.log ○ Links to transformers ● Spatial Log File ○ <myworkspace>_log.ffs
  • 14. More Tools for Troubleshooting in Workbench ● Logging - old school but effective ○ Reviewing your log file ○ Where is my log file? <working folder><myworkspace>.log ○ Links to transformers ● Spatial Log File ○ <myworkspace>_log.ffs
  • 15. More Tools for Troubleshooting in Workbench ● Logging - old school but effective ○ Reviewing your log file ○ Where is my log file? <working folder><myworkspace>.log ○ Links to transformers ● Spatial Log File ○ <myworkspace>_log.ffs ● Debug Logging ○ FME Options > Translation > Log Message Filter > Log Debug ○ Extra detail for complex errors ○ Good for debugging with the HTTPCaller, SQLExecutor, SQLCreator
  • 16. Simply your data: Reduce the volume of data ● Reduces run time ● Easier to find issues Handy Transformers and Parameters for Simplifying Data ● TestFilter ● Sampler ● Max. Features to Read / Feature Types to Read Simplify your data when authoring
  • 17. Where oh where did I make that change? Workbench Search 🔗 Workspace Compare 🔗 ● Compare the differences between two workspaces ● Merge the changes
  • 18. Isolating the issue Make a Small Example ● Save as Template ○ Save with Cache ● Recorder / Player ● Sampler
  • 19. Poll: Have you seen any new troubleshooting ideas that you might try?
  • 20. Debugging Scripts Python scripts can appear in various places in Workbench ● PythonCaller ● Startup/Shutdown Python Script ● Scripted Parameters Script troubleshooting approaches: ● Python print statements ● Logger transformer ● Build complex scripts in a separate Python editor and import into FME
  • 21. Many of the tips we have reviewed already also apply to FME Server Debugging on FME Server If a workspace doesnʼt run correctly on FME Server, try it on Desktop first If it works on Desktop, but not FME Server, look for differences in the environments ● Permissions ● Dependencies ○ File paths ○ Missing transformers ○ Missing Python modules ○ Missing Web and Database Connections ○ Mismatched FME Versions ● Nondescript errors ○ Look at fmeprocessmonitorengine.log
  • 23. Even more thoughts on this topic… Performance troubleshooting 🔗 Database Errors 🔗 ● Errors can be buried in a transaction ○ The entire transaction is rolled back ○ Try Transaction Interval = 1 ● Save As Workspace Template doesnʼt encompass database data ○ Recorder / Player ● Test SQL scripts in the appropriate database manager ● Database system log files
  • 24. Some Best Practices ● Versioning workspaces ○ Publishing to FME Server & Commit ○ Workspace Compare ● Good workspace practices help with maintenance 🔗 🎓 ○ Annotations / bookmarks ○ Look for repetition in your workspace - multiple readers / writers. Many Testers ○ Should you rename Transformers
  • 25. Summary So weʼve covered: ● What it looks like when you do encounter issues ● Tools for troubleshooting workspaces ● Testing and debugging in FME Server
  • 26. Resources ● Feature Caching🔗 ● Break Points🔗 ● Visual Preview 🔗 ● Workspace Compare 🔗 ● Workbench Best Practices 🔗 ● Debugging Workspaces on FME Server 🔗
  • 27. Claim Your Community Badge Get community badges for watching webinars! fme.ly/WebinarBadge Todayʼs Code: PSLGW
  • 28. Q&A
  • 29. Thank you! Free Trial | Upgrade Chat with us info@safe.com Connect with us in the Community Connect with us for more FME Please share your feedback with us through the webinar survey!