SlideShare a Scribd company logo
Creating Custom Solutions
with FME and Python
Meet the Presenters
Laura Wu Debbi Leung Kailin Opaleychuk
Agenda
What we’re covering today.
● Python Overview
● Integrating Python and FME
● Understanding Compatibility
● Intro to ArcPy
● Importing Modules
Python Overview
What is Python?
Python is a scripting language.
Why is Python attractive?
● Free
● Well documented
● Compatible with many GIS
applications
Sample Python Script
import os
os. startfile(r"C:Program FilesFME2018fmeworkbench.exe")
import os
os. startfile("C:Program FilesFME2018fmeworkbench.exe")
Integrating Python and FME
Reasons for using
Python in FME:
● To perform transformations or other
operations FME workbench doesn’t
support
● Leverage third party modules and libraries
● Call or run a workspace via script (outside
FME Workbench)
● Migrate ArcPy or other Python scripts
● Automate data tasks
● Or perhaps models or processes were
initially developed using scripts
Reasons for using
Python in FME:
● To perform transformations or other
operations FME workbench doesn’t
support
● Leverage third party modules and libraries
● Call or run a workspace via script (outside
FME Workbench)
● Migrate ArcPy or other Python scripts
● Automate data tasks
● Or perhaps models or processes were
initially developed using scripts
However…
There are numerous FME transformers for
performing almost every task you can think
of without resorting to scripting.
If you are thinking about writing a Python script to do
something in FME, you should make absolutely sure there
is not an existing transformer available already.
Where can I find
Python in FME
Workbench?
● Startup Python scripts
● Shutdown Python scripts
● Scripted parameters
● PythonCaller/PythonCreator
transformers
Startup Python Scripts in FME
These parameters allow a
user to specify a Python
script that is executed prior
to a translation.
● Check validity of
database connections
● Create a backup for
source data
● Adding custom content
to the logfile
DEMO
Shutdown Python Scripts in FME
Enables a user to specify a
Python script that is executed
after a translation.
● Send a notification
email if the translation
fails.
● Moving or making a
copy of data results
● Scan the log for a
particular message or
condition
DEMO
Scripted Parameters
Set User Parameter values
dynamically based on output
from a Python script.
These run before any Startup
Scripts in the Workspace.
DEMO
PythonCreator
Can be used to start off a workflow within FME.
Creates any number of features to be used
within downstream transformers.
Key use:
● Read from data formats that FME doesn’t
currently support
DEMO
PythonCaller
Used mid-stream within an FME workflow
Run a python script once per input feature
OR
Run a python script on groups of input features
Key uses:
● Perform tasks where a transformer isn’t
available
● Complete complex manipulations on lists
DEMO
Understanding Python
Compatibility
Supported Python Versions
FME 2021 supports Python 3.6-3.8 and provisionally supports
Python 2.7 depending on the operating system.
On Windows, FME also supports using the Python interpreter
included with ArcGIS products, which grants access to the
ArcPy library.
Different Python versions are supported depending on the FME
version used--check documentation or go to fmeobjects folder
in the FME install directory.
Preferred Python Interpreter
This per-user setting controls:
● Default Python Compatibility value for new
workspaces
● Interpreter loaded for FME Python’s
interactive shell
● FME Python’s pip utility
Can use Custom Python interpreter instead of
the interpreter bundled with FME by
specifying a custom interpreter with setting.
Note: Does not affect Python interpreter
choice for existing workspaces. Found under Tools (menu) > FME Options > Translation > Python
Interpreter.
Python Compatibility
This parameter specifies the level of compatibility your
workspace or custom transformer has with different
versions of Python.
It also determines the Python interpreter used by FME
when running a workspace.
This parameter can be found in the Navigator, under
Workspace Parameters > Scripting.
Parameter is workspace-specific.
Choosing a Python
Interpreter
● Recommendation: use Python 3
○ Python 2.7 is end-of-life
○ FME 2023 will drop Python
2.7 support
● Other considerations:
○ Dependencies
○ ArcGIS Desktop’s ArcPy is
Python 2.7 only
Python on FME Server
Engines uses a workspace’s Python
Compatibility parameter to determine
which Python interpreter should be
used.
If a currently-running engine has
loaded an incompatible interpreter, the
engine will be restarted automatically
to load a compatible interpreter.
Python in different Operating Systems
Windows
Uses bundled
interpreter by
default
Can use custom
Python interpreter
Linux
Uses system
interpreter
Cannot use
custom Python
interpreter
macOS
Uses bundled
interpreter
Cannot use
custom Python
interpreter
Using ArcPy with FME
Introduction to ArcPy
ArcPy is a package that provides geographic data analysis, data conversion, data
management, and map automation with Python.
FME supports working with Python 2.7 and Python 3.x versions of ArcPy on the Windows
platform.
Requires a licensed installation of ArcGIS Desktop, ArcPro, or ArcGIS Enterprise in order
to use with FME.
A workspace’s Python Compatibility parameter must be set to the appropriate Esri
ArcGIS Python options in order to work
DEMO
Importing Modules
Python Modules in FME
You can use anything from the Python Standard Library with no extra work
Simply import os, fileinput, statistics, etc to use them
For everything else, you must install them for the python interpreter used in your
Workspace.
Install Python Packages
Use pip - package installer for Python
fme.exe python -m pip install <
package_name> --target <package_destination_folder>
The key is placing the package files into the right location for FME to use them
On FME Desktop, this is your FME user folder:
C:Users<user>DocumentsFMEPluginsPython
On FME Server, they go into Resources:
EnginePluginspython
DEMO
Resources
● Tutorial: Python and FME Basics
● Python Scripted Parameters in FME
● Installing Python Packages to FME
Desktop
● Using Python with FME Server
● Python FME API Documentation
● Python Documentation
Thank you!
Please type in your questions to the Q&A panel
or ask them anytime at safe.com/community
Connect with us for more FME

More Related Content

What's hot

Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Databricks
 
Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?
Guido Schmutz
 
Flink vs. Spark
Flink vs. SparkFlink vs. Spark
Flink vs. Spark
Slim Baltagi
 
Integrating Apache Spark and NiFi for Data Lakes
Integrating Apache Spark and NiFi for Data LakesIntegrating Apache Spark and NiFi for Data Lakes
Integrating Apache Spark and NiFi for Data Lakes
DataWorks Summit/Hadoop Summit
 
Unified Stream and Batch Processing with Apache Flink
Unified Stream and Batch Processing with Apache FlinkUnified Stream and Batch Processing with Apache Flink
Unified Stream and Batch Processing with Apache Flink
DataWorks Summit/Hadoop Summit
 
Strata sf - Amundsen presentation
Strata sf - Amundsen presentationStrata sf - Amundsen presentation
Strata sf - Amundsen presentation
Tao Feng
 
Apache Flink in the Cloud-Native Era
Apache Flink in the Cloud-Native EraApache Flink in the Cloud-Native Era
Apache Flink in the Cloud-Native Era
Flink Forward
 
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
Apache Phoenix and HBase: Past, Present and Future of SQL over HBaseApache Phoenix and HBase: Past, Present and Future of SQL over HBase
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
DataWorks Summit/Hadoop Summit
 
【Redmine】ツールバーボタンを作ろう
【Redmine】ツールバーボタンを作ろう【Redmine】ツールバーボタンを作ろう
【Redmine】ツールバーボタンを作ろう
Kohei Nakamura
 
The top 3 challenges running multi-tenant Flink at scale
The top 3 challenges running multi-tenant Flink at scaleThe top 3 challenges running multi-tenant Flink at scale
The top 3 challenges running multi-tenant Flink at scale
Flink Forward
 
Airflow Intro-1.pdf
Airflow Intro-1.pdfAirflow Intro-1.pdf
Airflow Intro-1.pdf
BagustTriCahyo1
 
When to use python in FME
When to use python in FMEWhen to use python in FME
When to use python in FME
Daniela Perri
 
A brief introduction to Machine Learning
A brief introduction to Machine LearningA brief introduction to Machine Learning
A brief introduction to Machine Learning
Wen-Tien Chang
 
Spark streaming + kafka 0.10
Spark streaming + kafka 0.10Spark streaming + kafka 0.10
Spark streaming + kafka 0.10
Joan Viladrosa Riera
 
Welcome to the Flink Community!
Welcome to the Flink Community!Welcome to the Flink Community!
Welcome to the Flink Community!
Flink Forward
 
The Good, The Bad and The Ugly of Event Sourcing
The Good, The Bad and The Ugly of Event Sourcing The Good, The Bad and The Ugly of Event Sourcing
The Good, The Bad and The Ugly of Event Sourcing
Dennis Doomen
 
Google Cloud and Data Pipeline Patterns
Google Cloud and Data Pipeline PatternsGoogle Cloud and Data Pipeline Patterns
Google Cloud and Data Pipeline Patterns
Lynn Langit
 
QGISプログラミング入門 2016Osaka編
QGISプログラミング入門 2016Osaka編QGISプログラミング入門 2016Osaka編
QGISプログラミング入門 2016Osaka編
Kosuke Asahi
 
Iceberg: a fast table format for S3
Iceberg: a fast table format for S3Iceberg: a fast table format for S3
Iceberg: a fast table format for S3
DataWorks Summit
 
Custom Applications with Spark's RDD: Spark Summit East talk by Tejas Patil
Custom Applications with Spark's RDD: Spark Summit East talk by Tejas PatilCustom Applications with Spark's RDD: Spark Summit East talk by Tejas Patil
Custom Applications with Spark's RDD: Spark Summit East talk by Tejas Patil
Spark Summit
 

What's hot (20)

Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
 
Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?
 
Flink vs. Spark
Flink vs. SparkFlink vs. Spark
Flink vs. Spark
 
Integrating Apache Spark and NiFi for Data Lakes
Integrating Apache Spark and NiFi for Data LakesIntegrating Apache Spark and NiFi for Data Lakes
Integrating Apache Spark and NiFi for Data Lakes
 
Unified Stream and Batch Processing with Apache Flink
Unified Stream and Batch Processing with Apache FlinkUnified Stream and Batch Processing with Apache Flink
Unified Stream and Batch Processing with Apache Flink
 
Strata sf - Amundsen presentation
Strata sf - Amundsen presentationStrata sf - Amundsen presentation
Strata sf - Amundsen presentation
 
Apache Flink in the Cloud-Native Era
Apache Flink in the Cloud-Native EraApache Flink in the Cloud-Native Era
Apache Flink in the Cloud-Native Era
 
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
Apache Phoenix and HBase: Past, Present and Future of SQL over HBaseApache Phoenix and HBase: Past, Present and Future of SQL over HBase
Apache Phoenix and HBase: Past, Present and Future of SQL over HBase
 
【Redmine】ツールバーボタンを作ろう
【Redmine】ツールバーボタンを作ろう【Redmine】ツールバーボタンを作ろう
【Redmine】ツールバーボタンを作ろう
 
The top 3 challenges running multi-tenant Flink at scale
The top 3 challenges running multi-tenant Flink at scaleThe top 3 challenges running multi-tenant Flink at scale
The top 3 challenges running multi-tenant Flink at scale
 
Airflow Intro-1.pdf
Airflow Intro-1.pdfAirflow Intro-1.pdf
Airflow Intro-1.pdf
 
When to use python in FME
When to use python in FMEWhen to use python in FME
When to use python in FME
 
A brief introduction to Machine Learning
A brief introduction to Machine LearningA brief introduction to Machine Learning
A brief introduction to Machine Learning
 
Spark streaming + kafka 0.10
Spark streaming + kafka 0.10Spark streaming + kafka 0.10
Spark streaming + kafka 0.10
 
Welcome to the Flink Community!
Welcome to the Flink Community!Welcome to the Flink Community!
Welcome to the Flink Community!
 
The Good, The Bad and The Ugly of Event Sourcing
The Good, The Bad and The Ugly of Event Sourcing The Good, The Bad and The Ugly of Event Sourcing
The Good, The Bad and The Ugly of Event Sourcing
 
Google Cloud and Data Pipeline Patterns
Google Cloud and Data Pipeline PatternsGoogle Cloud and Data Pipeline Patterns
Google Cloud and Data Pipeline Patterns
 
QGISプログラミング入門 2016Osaka編
QGISプログラミング入門 2016Osaka編QGISプログラミング入門 2016Osaka編
QGISプログラミング入門 2016Osaka編
 
Iceberg: a fast table format for S3
Iceberg: a fast table format for S3Iceberg: a fast table format for S3
Iceberg: a fast table format for S3
 
Custom Applications with Spark's RDD: Spark Summit East talk by Tejas Patil
Custom Applications with Spark's RDD: Spark Summit East talk by Tejas PatilCustom Applications with Spark's RDD: Spark Summit East talk by Tejas Patil
Custom Applications with Spark's RDD: Spark Summit East talk by Tejas Patil
 

Similar to Creating Custom Solutions with FME and Python

Creating Custom Solutions with FME and Python
Creating Custom Solutions with FME and PythonCreating Custom Solutions with FME and Python
Creating Custom Solutions with FME and Python
Safe Software
 
Beyond 49x Transformers: Don't be afraid of (the) Python!
Beyond 49x Transformers: Don't be afraid of (the) Python!Beyond 49x Transformers: Don't be afraid of (the) Python!
Beyond 49x Transformers: Don't be afraid of (the) Python!
Safe Software
 
Introduction to Python Unit -1 Part .pdf
Introduction to Python Unit -1 Part .pdfIntroduction to Python Unit -1 Part .pdf
Introduction to Python Unit -1 Part .pdf
VaibhavKumarSinghkal
 
Build and deploy scientific Python Applications
Build and deploy scientific Python Applications  Build and deploy scientific Python Applications
Build and deploy scientific Python Applications
Ramakrishna Reddy
 
Research paper on python by Rj
Research paper on python by RjResearch paper on python by Rj
Research paper on python by Rj
Shree M.L.Kakadiya MCA mahila college, Amreli
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docx
Ramakrishna Reddy Bijjam
 
Python quick guide1
Python quick guide1Python quick guide1
Python quick guide1
Kanchilug
 
Python Programming.pdf
Python Programming.pdfPython Programming.pdf
Python Programming.pdf
ssuser9a6ca1
 
openmp
openmpopenmp
openmp
Neel Bhad
 
Python
PythonPython
Python for students step by step guidance
Python for students step by step guidancePython for students step by step guidance
Python for students step by step guidance
MantoshKumar79
 
Module1-Chapter1_ppt.pptx
Module1-Chapter1_ppt.pptxModule1-Chapter1_ppt.pptx
Module1-Chapter1_ppt.pptx
SandeepR95
 
Getting Started with PHP Extensions
Getting Started with PHP ExtensionsGetting Started with PHP Extensions
Getting Started with PHP Extensions
MichaelBrunoLochemem
 
An Introduction to PyPy
An Introduction to PyPyAn Introduction to PyPy
An Introduction to PyPy
Michael Hudson-Doyle
 
Python Programming and ApplicationsUnit-1.docx
Python Programming and ApplicationsUnit-1.docxPython Programming and ApplicationsUnit-1.docx
Python Programming and ApplicationsUnit-1.docx
Manohar k
 
01 python introduction
01 python introduction 01 python introduction
01 python introduction
Tamer Ahmed Farrag, PhD
 
Introduction to Python Programming Basics
Introduction  to  Python  Programming BasicsIntroduction  to  Python  Programming Basics
Introduction to Python Programming Basics
Dhana malar
 
Exploring Five Lesser-Known Python Libraries
Exploring Five Lesser-Known Python LibrariesExploring Five Lesser-Known Python Libraries
Exploring Five Lesser-Known Python Libraries
MinhazulAbedin27
 
Introduction to Apache Beam
Introduction to Apache BeamIntroduction to Apache Beam
Introduction to Apache Beam
Knoldus Inc.
 
Pyhton-1a-Basics.pdf
Pyhton-1a-Basics.pdfPyhton-1a-Basics.pdf
Pyhton-1a-Basics.pdf
Mattupallipardhu
 

Similar to Creating Custom Solutions with FME and Python (20)

Creating Custom Solutions with FME and Python
Creating Custom Solutions with FME and PythonCreating Custom Solutions with FME and Python
Creating Custom Solutions with FME and Python
 
Beyond 49x Transformers: Don't be afraid of (the) Python!
Beyond 49x Transformers: Don't be afraid of (the) Python!Beyond 49x Transformers: Don't be afraid of (the) Python!
Beyond 49x Transformers: Don't be afraid of (the) Python!
 
Introduction to Python Unit -1 Part .pdf
Introduction to Python Unit -1 Part .pdfIntroduction to Python Unit -1 Part .pdf
Introduction to Python Unit -1 Part .pdf
 
Build and deploy scientific Python Applications
Build and deploy scientific Python Applications  Build and deploy scientific Python Applications
Build and deploy scientific Python Applications
 
Research paper on python by Rj
Research paper on python by RjResearch paper on python by Rj
Research paper on python by Rj
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docx
 
Python quick guide1
Python quick guide1Python quick guide1
Python quick guide1
 
Python Programming.pdf
Python Programming.pdfPython Programming.pdf
Python Programming.pdf
 
openmp
openmpopenmp
openmp
 
Python
PythonPython
Python
 
Python for students step by step guidance
Python for students step by step guidancePython for students step by step guidance
Python for students step by step guidance
 
Module1-Chapter1_ppt.pptx
Module1-Chapter1_ppt.pptxModule1-Chapter1_ppt.pptx
Module1-Chapter1_ppt.pptx
 
Getting Started with PHP Extensions
Getting Started with PHP ExtensionsGetting Started with PHP Extensions
Getting Started with PHP Extensions
 
An Introduction to PyPy
An Introduction to PyPyAn Introduction to PyPy
An Introduction to PyPy
 
Python Programming and ApplicationsUnit-1.docx
Python Programming and ApplicationsUnit-1.docxPython Programming and ApplicationsUnit-1.docx
Python Programming and ApplicationsUnit-1.docx
 
01 python introduction
01 python introduction 01 python introduction
01 python introduction
 
Introduction to Python Programming Basics
Introduction  to  Python  Programming BasicsIntroduction  to  Python  Programming Basics
Introduction to Python Programming Basics
 
Exploring Five Lesser-Known Python Libraries
Exploring Five Lesser-Known Python LibrariesExploring Five Lesser-Known Python Libraries
Exploring Five Lesser-Known Python Libraries
 
Introduction to Apache Beam
Introduction to Apache BeamIntroduction to Apache Beam
Introduction to Apache Beam
 
Pyhton-1a-Basics.pdf
Pyhton-1a-Basics.pdfPyhton-1a-Basics.pdf
Pyhton-1a-Basics.pdf
 

More from Safe Software

Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Safe Software
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
Safe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
Safe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Safe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Safe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Safe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
Safe Software
 

More from Safe Software (20)

Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 

Recently uploaded

Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Neo4j
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
mz5nrf0n
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
timtebeek1
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
TheSMSPoint
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Octavian Nadolu
 
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise EditionWhy Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Envertis Software Solutions
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
Remote DBA Services
 
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppAI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
Google
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptxLORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
lorraineandreiamcidl
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
E-commerce Application Development Company.pdf
E-commerce Application Development Company.pdfE-commerce Application Development Company.pdf
E-commerce Application Development Company.pdf
Hornet Dynamics
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
Yara Milbes
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
Ayan Halder
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
rodomar2
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j
 

Recently uploaded (20)

Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
 
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise EditionWhy Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
 
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppAI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptxLORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
E-commerce Application Development Company.pdf
E-commerce Application Development Company.pdfE-commerce Application Development Company.pdf
E-commerce Application Development Company.pdf
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
 

Creating Custom Solutions with FME and Python

  • 2. Meet the Presenters Laura Wu Debbi Leung Kailin Opaleychuk
  • 3. Agenda What we’re covering today. ● Python Overview ● Integrating Python and FME ● Understanding Compatibility ● Intro to ArcPy ● Importing Modules
  • 5. What is Python? Python is a scripting language. Why is Python attractive? ● Free ● Well documented ● Compatible with many GIS applications
  • 6. Sample Python Script import os os. startfile(r"C:Program FilesFME2018fmeworkbench.exe") import os os. startfile("C:Program FilesFME2018fmeworkbench.exe")
  • 8. Reasons for using Python in FME: ● To perform transformations or other operations FME workbench doesn’t support ● Leverage third party modules and libraries ● Call or run a workspace via script (outside FME Workbench) ● Migrate ArcPy or other Python scripts ● Automate data tasks ● Or perhaps models or processes were initially developed using scripts
  • 9. Reasons for using Python in FME: ● To perform transformations or other operations FME workbench doesn’t support ● Leverage third party modules and libraries ● Call or run a workspace via script (outside FME Workbench) ● Migrate ArcPy or other Python scripts ● Automate data tasks ● Or perhaps models or processes were initially developed using scripts However…
  • 10. There are numerous FME transformers for performing almost every task you can think of without resorting to scripting. If you are thinking about writing a Python script to do something in FME, you should make absolutely sure there is not an existing transformer available already.
  • 11. Where can I find Python in FME Workbench? ● Startup Python scripts ● Shutdown Python scripts ● Scripted parameters ● PythonCaller/PythonCreator transformers
  • 12. Startup Python Scripts in FME These parameters allow a user to specify a Python script that is executed prior to a translation. ● Check validity of database connections ● Create a backup for source data ● Adding custom content to the logfile
  • 13. DEMO
  • 14. Shutdown Python Scripts in FME Enables a user to specify a Python script that is executed after a translation. ● Send a notification email if the translation fails. ● Moving or making a copy of data results ● Scan the log for a particular message or condition
  • 15. DEMO
  • 16. Scripted Parameters Set User Parameter values dynamically based on output from a Python script. These run before any Startup Scripts in the Workspace.
  • 17. DEMO
  • 18. PythonCreator Can be used to start off a workflow within FME. Creates any number of features to be used within downstream transformers. Key use: ● Read from data formats that FME doesn’t currently support
  • 19. DEMO
  • 20. PythonCaller Used mid-stream within an FME workflow Run a python script once per input feature OR Run a python script on groups of input features Key uses: ● Perform tasks where a transformer isn’t available ● Complete complex manipulations on lists
  • 21. DEMO
  • 23. Supported Python Versions FME 2021 supports Python 3.6-3.8 and provisionally supports Python 2.7 depending on the operating system. On Windows, FME also supports using the Python interpreter included with ArcGIS products, which grants access to the ArcPy library. Different Python versions are supported depending on the FME version used--check documentation or go to fmeobjects folder in the FME install directory.
  • 24. Preferred Python Interpreter This per-user setting controls: ● Default Python Compatibility value for new workspaces ● Interpreter loaded for FME Python’s interactive shell ● FME Python’s pip utility Can use Custom Python interpreter instead of the interpreter bundled with FME by specifying a custom interpreter with setting. Note: Does not affect Python interpreter choice for existing workspaces. Found under Tools (menu) > FME Options > Translation > Python Interpreter.
  • 25. Python Compatibility This parameter specifies the level of compatibility your workspace or custom transformer has with different versions of Python. It also determines the Python interpreter used by FME when running a workspace. This parameter can be found in the Navigator, under Workspace Parameters > Scripting. Parameter is workspace-specific.
  • 26. Choosing a Python Interpreter ● Recommendation: use Python 3 ○ Python 2.7 is end-of-life ○ FME 2023 will drop Python 2.7 support ● Other considerations: ○ Dependencies ○ ArcGIS Desktop’s ArcPy is Python 2.7 only
  • 27. Python on FME Server Engines uses a workspace’s Python Compatibility parameter to determine which Python interpreter should be used. If a currently-running engine has loaded an incompatible interpreter, the engine will be restarted automatically to load a compatible interpreter.
  • 28. Python in different Operating Systems Windows Uses bundled interpreter by default Can use custom Python interpreter Linux Uses system interpreter Cannot use custom Python interpreter macOS Uses bundled interpreter Cannot use custom Python interpreter
  • 30. Introduction to ArcPy ArcPy is a package that provides geographic data analysis, data conversion, data management, and map automation with Python. FME supports working with Python 2.7 and Python 3.x versions of ArcPy on the Windows platform. Requires a licensed installation of ArcGIS Desktop, ArcPro, or ArcGIS Enterprise in order to use with FME. A workspace’s Python Compatibility parameter must be set to the appropriate Esri ArcGIS Python options in order to work
  • 31. DEMO
  • 33. Python Modules in FME You can use anything from the Python Standard Library with no extra work Simply import os, fileinput, statistics, etc to use them For everything else, you must install them for the python interpreter used in your Workspace.
  • 34. Install Python Packages Use pip - package installer for Python fme.exe python -m pip install < package_name> --target <package_destination_folder> The key is placing the package files into the right location for FME to use them On FME Desktop, this is your FME user folder: C:Users<user>DocumentsFMEPluginsPython On FME Server, they go into Resources: EnginePluginspython
  • 35. DEMO
  • 36. Resources ● Tutorial: Python and FME Basics ● Python Scripted Parameters in FME ● Installing Python Packages to FME Desktop ● Using Python with FME Server ● Python FME API Documentation ● Python Documentation
  • 37. Thank you! Please type in your questions to the Q&A panel or ask them anytime at safe.com/community Connect with us for more FME