SlideShare a Scribd company logo
1 of 59
Download to read offline
Geospatial Synergy:
Amplifying Efficiency with
FME & Esri
Agenda
1 Introduction
2 Safe & FME
3 Jan, Locus: Optimizing Bulk Data Updates in ArcGIS Online
4 James, 1Spatial: Model build ArcPy - FME
5 Resources & Next Steps
6 Q&A
Agenda
Welcome to Livestorm.
A few ways to engage with us during the webinar:
Audio issues? Click this for 4 simple
troubleshooting steps.
1
Introduction
Explore the remarkable possibilities
that open up when FME and Esri
technologies converge.
And manage your geospatial data with
unprecedented efficiency.
Introduction
One platform, two technologies
FME Form FME Flow
Build and run data workflows Automate data workflows
FME Flow Hosted
Safe Software managed instance
fme.safe.com/platform
FME Enterprise Integration Platform
Safe & FME
3
Optimizing
Bulk Data
Updates in
ArcGIS Online
Jan
Roggisch
GIS & System Integration Consultant
Locus Limited
Optimizing Bulk Data Updates in ArcGIS Online
Agenda
1. Who is the client?
2. What is their challenge?
3. How did Locus help?
4. Conclusion
The client:
Auckland
Council
Auckland Council
(Org.)
https://en.wikipedia.org/wiki/Auckland_Region#/map/0
● Established in 2010
● Service area ~5,000 km2
● Population >1.6 million
● ~12,000 staff
● Annual budget $3 billion
● Largest council in Oceania
Optimizing Bulk Data Updates in ArcGIS Online
Auckland Council (GIS)
● ArcGIS-based public maps & apps
● Datasets are
○ diverse
○ complex
○ large
○ Dynamic
● Need for scalability and resilience (e.g. 2023 Floods)
Optimizing Bulk Data Updates in ArcGIS Online
Auckland Anniversary Weekend Floods
Newsroo
m
Richard Easther/Twitter
NZ Herald
Optimizing Bulk Data Updates in ArcGIS Online
The
challenge:
Provide
highly-available
apps
First Steps
No
infrastructure
maintenance
Scalability,
Resilience,
Security built-in
Upgrades
taken care of
Duration
service outage
Timing
after-hours
work
Dependent on
on-premise
data
processes
Optimizing Bulk Data Updates in ArcGIS Online
Host data and apps in ArcGIS Online Challenges around data updates
Hosted view
● Requires identical FGDBs (schema)
● Manually update FDGBs and services
● Manually switch view source
Optimizing Bulk Data Updates in ArcGIS Online
Optimizing Bulk Data Updates in ArcGIS Online
Hosted view
● Requires identical FGDBs (schema)
● Manually update FDGBs and services
● Manually switch view source
Optimizing Bulk Data Updates in ArcGIS Online
The solution:
Automation
What to do?
● ArcGIS Admin REST API
● Esri documentation
● Excel-pattern
● Trial & Error
Optimizing Bulk Data Updates in ArcGIS Online
The result – FGDB replacement
● 61 transformers
● 7 x HttpCaller
● JsonX-Former
● Dedicated error stream
Optimizing Bulk Data Updates in ArcGIS Online
The result – data source swap
● 67 transformers
● 8 x HttpCaller
● JsonX-Former
● Dedicated error stream
Optimizing Bulk Data Updates in ArcGIS Online
The result – master workspace
Optimizing Bulk Data Updates in ArcGIS Online
Conclusion
1. Data updates fully automated
2. Downtime reduced to seconds
3. Improved public service delivery
Recap
Conclusion
Thank you!
jan.roggisch@locus.co.nz
4
Model build
ArcPy - FME
James
Botterill
Senior Consultant
1Spatial Australia
Model build ArcPy - FME
Agenda
1. Species of National Significance
2. Case Study – Raster Gridding
3. Vector to Raster conversion
4. ESRI vs FME raster gridding tools
5. ArcPy within FME – featureclass processing
6. Tips & Tricks
Biodiversity
Changes to the landscape and native habitat
as a result of human activity have put many
species at risk of extinction.
Ecological communities are unique and
naturally occurring groups of plants and
animals. Their presence can be determined
by factors such as soil type, position in the
landscape, climate and water availability.
Environment Protection and Biodiversity Act
1999 (EPBC Act) protects Australia's native
species and ecological communities.
• Identification and listing of species and
ecological communities as threatened
• Development of conservation
advice and recovery plans for listed species
and ecological communities
• Register of critical habitat
• Recognition of key threatening processes
• Where appropriate, reducing the impacts of
these processes through threat abatement
plans and non-statutory threat abatement
advices
Model build ArcPy - FME
Threatened Ecological
Communities of Australia
(dcceew.gov.au)
Threatened
Species &
Ecological
Communities
of National
Environmental
Significance
Species of National
Significance Information as listed in the Environment
Protection and Biodiversity Act 1999 (EPBC act)
1. Ecological community name
2. Ecological threatened status
○ Critically Endangered, Endangered, Vulnerable or
Conservation Dependent
3. Indicative occurrence (known and predicted
areas) produced by spatial ecologists
4. Cell size for public access
○ 1 km grid resolution (0.01°) or
○ ~10km for species classed as (SNES) sensitive by
respective States and Territories.
5. Links to further information in the Species
Profile and Threats Database (SPRAT)
Model build ArcPy - FME
Why Generalize?
Data source: Australia - Species of National Environmental Significance Distributions (public grids) | Find Environmental Data (dcceew.gov.au)
To provide information to the community about Australia’s protected species, and in line with the
Government’s policy of open data access, the Department’s threatened and migratory
species distributions have been generalized to 1km and 10km
The generalized product is aimed at addressing concerns regarding the release of detailed
locations of species sensitive to illegal collection and disturbance while still providing public
access to the distributions of threatened species.
Indicative occurrence - coded to indicate species presence (pres_rank) with:
1 ‘Species or species habitat likely to occur’ and
2 ‘Species or species habitat may occur’.
Model build ArcPy - FME
Processing Species data
Department was seeking to improve spatial generalization methods to process Species
Distributions
● Current automated python scripts are difficult to maintain.
● Manual steps and QA checks run several times per week to keep it up to date
● Complex set of business rules dictate which species are to use private or public grids…
● Hiding details for sensitive species
Can an FME workflow replace or support the above requirements?
Model build ArcPy - FME
Model build ArcPy - FME
Species
Distribution
Polygons
SPRAT changes
(insert, update, delete)
Public
1km
Public
10km
CHANGEDATE WORK_PERFORMED FME_DB_OPERATION
20220811 Deyeuxia ramosa (87970) - SEAP new distribution INSERT
20220811 Nematolepis rhytidophylla (64936) - SEAP updated distribution INSERT
20220811 Pherosphaera fitzgeraldii (40324) - SEAP updated distribution UPDATE
20220811 Syncomistes rastellus (88733) - SEAP new distribution INSERT
(filter)
Individual
Taxon
cell assignment type = maximum area
pres_rank = gridcode
priority = 100-gridcode
cell_size = 0.01, 0.1
Generalisation
1. Vector to Raster 2. Raster to Vector
snap
raster
(assign)
Sensitive
Species
46
36
36
36
36
26
Original
Polygon
26
26
1km Raster
Grid
10km Raster Grid
Generalised
Polygon
ESRI Gridding
Existing approach….
Polygon to raster conversion
• Cell assignment Maximum Area
• Priority is specified
(100-Presence_Rank) e.g., 100 minus 26 = 74.
The small “Known” presence given largest cell yields
due to high 74 priority
• Environment Snap raster used as a
template raster grid (next slide)
64
64
64
64
74
54
64
74
54
54
74
64
64
64
64
Model build ArcPy - FME
Snap raster reference
An environment used to
snap or align an extent
during execution
Snapping usually results in a
larger output extent than the
given extent.
Data source:
https://pro.arcgis.com/en/pro-app/latest/tool-reference/environment-setti
ngs/how-snap-raster-environment-works.htm
Figure (b) shows the snapped extent after execution.
Figure (a) shows the extent to be snapped.
74
Model build ArcPy - FME
FME Gridding
Test alternative approaches….
Raster gridding transformers
• NumericRasteriser - creates single-band numeric raster. Precede
with 3DForcer for Priority and followed by RasterMosaicker where
cell assignment (Overlapping Values) = maximum
Summary:
• use RasterPropertyExtractor after reading snapraster dataset
• 1-1 featuremerger with features to process
• NumRaster to use values @round(<ground extents>) and cell spacing
• slightly slower than ArcGIS
• grid extents end up larger
• a dissolve is required as secondary step for cell assignment
Model build ArcPy - FME
ArcPy within
FME
Model build ArcPy - FME
ArcPRO - Export to Python
Both model builder and the
Geoprocessing “History” have an
option to export Python script
Data Source:
https://pro.arcgis.com/en/pro-app/latest/help/analysis/geoprocessing/modelbuilder/exporting-a-model-to-python.htm
Model build ArcPy - FME
Feature Class Processing - ArcPy in FME
1. Expose _dataset or use a PATH reader
2. AttributeCreator for names of feature
classes and file paths for input & output
feature classes
3. Run the ArcPy operation on incoming
feature class using the PythonCaller
transformer
4. Read the results back into the FME
workspace with the FeatureReader
transformer
Model build ArcPy - FME
Cell size '0.1', '0.01', '0.001’
equates to 10km, 1km, 100m
Summary
Vector-Raster
Raster to Vector
Input feature
Prerequisites: ArcPRO license and spatial analyst extension
Set workspace - _dataset is from working_gdb FeatureReader
Vector to raster – maximum cell, snapraster, priority
Raster to vector – using output paths
FME Tips using Python
• Set environments to working geodatabase + overwrite output true
• Variables for workspace
cell size = feature.getAttribute(‘_path%’)
arcpy.env.snapraster cell_size
• Replace(‘’, ‘’) if any spaces in paths
• FMElogfile() concatenate attributes in a notification string
and inform geoprocessing progress in the transaction log.
• Feature cache mode needs to be emptied
To avoid locks, errors 160001-170000 table exists, click the overwrite geodatabase in full
Data Source: https://community.safe.com/s/article/using-arcpy-for-fme-feature-processing
Model build ArcPy - FME
Conclusion
FME workflow with ArcGeoprocessing
OUTPUTS
fileGDB
SNES Public
Species with
metadata
SDE
SNES
Public
Species
QA report
CHILD WORKSPACE
INPUTS
Workspace
Runner
metadata
Last load date
SPRAT
Find all the
additions/updates
to TaxonIDs
Generalising
ArcPy – Polygon-Raster-Polygon
generaliser_working.gdb & snap raster
Process
individual
Taxons
Species Grids
Business
rules
Sensitive Species Resolution
Synonymy
wkhtmltopdf
PARENT WORKSPACE
Model build ArcPy - FME
Summary
• ArcGIS gridding functionality & approach
were most consistent to the generalization
requirements for processing species
distribution grids.
• FME superior in terms of orchestrating the
integration visually. Where FME transformers
lack ESRI functionality, incorporate ArcPy to
do the geoprocessing mid workflow.
• Get out and support local conservation
efforts!
Model build ArcPy - FME
Resources
• DCCEEW www.dcceew.gov.au/environment/biodiversity/threatened
• https://community.safe.com/s/article/notes-on-fme-and-esri-versions-and-compatibility
• ESRI How Polygon To Raster works—ArcGIS Pro
• ESRI ArcPro snap raster
• FME NumericRasteriser Popularity = 167/468
• Using Arcpy for FME Feature Processing
Model build ArcPy - FME
Thank you!
james.botterill@1spatial.com
5
Resources &
Next Steps
Get our Ebook
Spatial Data for the
Enterprise
fme.ly/gzc
Guided learning experiences
at your fingertips
community.safe.com
/s/academy
FME Academy
Resources & Next Steps
Check out how-to’s & demos
in the knowledge base
community.safe.com
/s/knowledge-base
Knowledge Base Webinars
Upcoming & on-demand
webinars
safe.com/webinars
Resources & Next Steps
We’d love to help you get
started.
Get in touch with us at
info@safe.com
Experience the FME Accelerator
Contact Us
Unlock the power of your
data in only 90 minutes
Register for free at
fme.safe.com/accelerator
Resources & Next Steps
ClaimYour Community Badge &
Dive into the new Community!
● Get community badges for watching
webinars
● community.safe.com
● Today’s code: APWFG
Join the Community today!
6
Q&A
ThankYou
Recap of Next Steps
1 Join the FME Community
2 Contact James: james.botterill@1spatial.com & Jan: jan.roggisch@locus.co.nz
3 Experience the FME Accelerator
Please fill out our webinar survey

More Related Content

Similar to Geospatial Synergy: Amplifying Efficiency with FME & Esri

Development and Applications of Distributed IoT Sensors for Intermittent Conn...
Development and Applications of Distributed IoT Sensors for Intermittent Conn...Development and Applications of Distributed IoT Sensors for Intermittent Conn...
Development and Applications of Distributed IoT Sensors for Intermittent Conn...InfluxData
 
Sparc的未来发展方向
Sparc的未来发展方向Sparc的未来发展方向
Sparc的未来发展方向legendnet368
 
Memory-Driven Near-Data Acceleration and its application to DOME/SKA
 Memory-Driven Near-Data Acceleration and its application to DOME/SKA Memory-Driven Near-Data Acceleration and its application to DOME/SKA
Memory-Driven Near-Data Acceleration and its application to DOME/SKAinside-BigData.com
 
byteLAKE's expertise across NVIDIA architectures and configurations
byteLAKE's expertise across NVIDIA architectures and configurationsbyteLAKE's expertise across NVIDIA architectures and configurations
byteLAKE's expertise across NVIDIA architectures and configurationsbyteLAKE
 
Scaling Open Source Big Data Cloud Applications is Easy/Hard
Scaling Open Source Big Data Cloud Applications is Easy/HardScaling Open Source Big Data Cloud Applications is Easy/Hard
Scaling Open Source Big Data Cloud Applications is Easy/HardPaul Brebner
 
Metadata and Provenance for ML Pipelines with Hopsworks
Metadata and Provenance for ML Pipelines with Hopsworks Metadata and Provenance for ML Pipelines with Hopsworks
Metadata and Provenance for ML Pipelines with Hopsworks Jim Dowling
 
Blue Waters and Resource Management - Now and in the Future
 Blue Waters and Resource Management - Now and in the Future Blue Waters and Resource Management - Now and in the Future
Blue Waters and Resource Management - Now and in the Futureinside-BigData.com
 
Times ten 18.1_overview_meetup
Times ten 18.1_overview_meetupTimes ten 18.1_overview_meetup
Times ten 18.1_overview_meetupByung Ho Lee
 
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facilityinside-BigData.com
 
byteLAKE's Alveo FPGA Solutions
byteLAKE's Alveo FPGA SolutionsbyteLAKE's Alveo FPGA Solutions
byteLAKE's Alveo FPGA SolutionsbyteLAKE
 
Monitoring federation open stack infrastructure
Monitoring federation open stack infrastructureMonitoring federation open stack infrastructure
Monitoring federation open stack infrastructureFernando Lopez Aguilar
 
Spark Development Lifecycle at Workday - ApacheCon 2020
Spark Development Lifecycle at Workday - ApacheCon 2020Spark Development Lifecycle at Workday - ApacheCon 2020
Spark Development Lifecycle at Workday - ApacheCon 2020Pavel Hardak
 
Apache Spark Development Lifecycle @ Workday - ApacheCon 2020
Apache Spark Development Lifecycle @ Workday - ApacheCon 2020Apache Spark Development Lifecycle @ Workday - ApacheCon 2020
Apache Spark Development Lifecycle @ Workday - ApacheCon 2020Eren Avşaroğulları
 
times ten in-memory database for extreme performance
times ten in-memory database for extreme performancetimes ten in-memory database for extreme performance
times ten in-memory database for extreme performanceOracle Korea
 
Big data at experimental facilities
Big data at experimental facilitiesBig data at experimental facilities
Big data at experimental facilitiesIan Foster
 
Kickstart your Kafka with Faker Data | Francesco Tisiot, Aiven.io
Kickstart your Kafka with Faker Data | Francesco Tisiot, Aiven.ioKickstart your Kafka with Faker Data | Francesco Tisiot, Aiven.io
Kickstart your Kafka with Faker Data | Francesco Tisiot, Aiven.ioHostedbyConfluent
 
FME World Tour 2015 - Around the World - Ken Bragg
FME World Tour 2015 - Around the World - Ken BraggFME World Tour 2015 - Around the World - Ken Bragg
FME World Tour 2015 - Around the World - Ken BraggIMGS
 

Similar to Geospatial Synergy: Amplifying Efficiency with FME & Esri (20)

Development and Applications of Distributed IoT Sensors for Intermittent Conn...
Development and Applications of Distributed IoT Sensors for Intermittent Conn...Development and Applications of Distributed IoT Sensors for Intermittent Conn...
Development and Applications of Distributed IoT Sensors for Intermittent Conn...
 
Sparc的未来发展方向
Sparc的未来发展方向Sparc的未来发展方向
Sparc的未来发展方向
 
Memory-Driven Near-Data Acceleration and its application to DOME/SKA
 Memory-Driven Near-Data Acceleration and its application to DOME/SKA Memory-Driven Near-Data Acceleration and its application to DOME/SKA
Memory-Driven Near-Data Acceleration and its application to DOME/SKA
 
byteLAKE's expertise across NVIDIA architectures and configurations
byteLAKE's expertise across NVIDIA architectures and configurationsbyteLAKE's expertise across NVIDIA architectures and configurations
byteLAKE's expertise across NVIDIA architectures and configurations
 
Scaling Open Source Big Data Cloud Applications is Easy/Hard
Scaling Open Source Big Data Cloud Applications is Easy/HardScaling Open Source Big Data Cloud Applications is Easy/Hard
Scaling Open Source Big Data Cloud Applications is Easy/Hard
 
EOSC-hub & Geohazards TEP
EOSC-hub & Geohazards TEPEOSC-hub & Geohazards TEP
EOSC-hub & Geohazards TEP
 
Metadata and Provenance for ML Pipelines with Hopsworks
Metadata and Provenance for ML Pipelines with Hopsworks Metadata and Provenance for ML Pipelines with Hopsworks
Metadata and Provenance for ML Pipelines with Hopsworks
 
Blue Waters and Resource Management - Now and in the Future
 Blue Waters and Resource Management - Now and in the Future Blue Waters and Resource Management - Now and in the Future
Blue Waters and Resource Management - Now and in the Future
 
Times ten 18.1_overview_meetup
Times ten 18.1_overview_meetupTimes ten 18.1_overview_meetup
Times ten 18.1_overview_meetup
 
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
 
byteLAKE's Alveo FPGA Solutions
byteLAKE's Alveo FPGA SolutionsbyteLAKE's Alveo FPGA Solutions
byteLAKE's Alveo FPGA Solutions
 
Monitoring federation open stack infrastructure
Monitoring federation open stack infrastructureMonitoring federation open stack infrastructure
Monitoring federation open stack infrastructure
 
Resume Maumita Biswas
Resume Maumita BiswasResume Maumita Biswas
Resume Maumita Biswas
 
Spark Development Lifecycle at Workday - ApacheCon 2020
Spark Development Lifecycle at Workday - ApacheCon 2020Spark Development Lifecycle at Workday - ApacheCon 2020
Spark Development Lifecycle at Workday - ApacheCon 2020
 
Apache Spark Development Lifecycle @ Workday - ApacheCon 2020
Apache Spark Development Lifecycle @ Workday - ApacheCon 2020Apache Spark Development Lifecycle @ Workday - ApacheCon 2020
Apache Spark Development Lifecycle @ Workday - ApacheCon 2020
 
times ten in-memory database for extreme performance
times ten in-memory database for extreme performancetimes ten in-memory database for extreme performance
times ten in-memory database for extreme performance
 
Big data at experimental facilities
Big data at experimental facilitiesBig data at experimental facilities
Big data at experimental facilities
 
Kickstart your Kafka with Faker Data | Francesco Tisiot, Aiven.io
Kickstart your Kafka with Faker Data | Francesco Tisiot, Aiven.ioKickstart your Kafka with Faker Data | Francesco Tisiot, Aiven.io
Kickstart your Kafka with Faker Data | Francesco Tisiot, Aiven.io
 
System mldl meetup
System mldl meetupSystem mldl meetup
System mldl meetup
 
FME World Tour 2015 - Around the World - Ken Bragg
FME World Tour 2015 - Around the World - Ken BraggFME World Tour 2015 - Around the World - Ken Bragg
FME World Tour 2015 - Around the World - Ken Bragg
 

More from Safe Software

Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemSafe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISSafe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersSafe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsSafe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategySafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Safe Software
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMESafe Software
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Safe Software
 
Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Safe Software
 
Bringing Open Data Integration to the (SWECO) People
Bringing Open Data Integration to the (SWECO) PeopleBringing Open Data Integration to the (SWECO) People
Bringing Open Data Integration to the (SWECO) PeopleSafe Software
 
Democratizing Transformer Creation in FME
Democratizing Transformer Creation in FMEDemocratizing Transformer Creation in FME
Democratizing Transformer Creation in FMESafe Software
 

More from Safe Software (20)

Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FME
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...
 
Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework
 
Bringing Open Data Integration to the (SWECO) People
Bringing Open Data Integration to the (SWECO) PeopleBringing Open Data Integration to the (SWECO) People
Bringing Open Data Integration to the (SWECO) People
 
Democratizing Transformer Creation in FME
Democratizing Transformer Creation in FMEDemocratizing Transformer Creation in FME
Democratizing Transformer Creation in FME
 

Recently uploaded

Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 

Recently uploaded (20)

Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 

Geospatial Synergy: Amplifying Efficiency with FME & Esri

  • 2. Agenda 1 Introduction 2 Safe & FME 3 Jan, Locus: Optimizing Bulk Data Updates in ArcGIS Online 4 James, 1Spatial: Model build ArcPy - FME 5 Resources & Next Steps 6 Q&A Agenda
  • 3. Welcome to Livestorm. A few ways to engage with us during the webinar: Audio issues? Click this for 4 simple troubleshooting steps.
  • 5. Explore the remarkable possibilities that open up when FME and Esri technologies converge. And manage your geospatial data with unprecedented efficiency. Introduction
  • 6. One platform, two technologies FME Form FME Flow Build and run data workflows Automate data workflows FME Flow Hosted Safe Software managed instance fme.safe.com/platform FME Enterprise Integration Platform Safe & FME
  • 8. Jan Roggisch GIS & System Integration Consultant Locus Limited Optimizing Bulk Data Updates in ArcGIS Online
  • 9. Agenda 1. Who is the client? 2. What is their challenge? 3. How did Locus help? 4. Conclusion
  • 11. Auckland Council (Org.) https://en.wikipedia.org/wiki/Auckland_Region#/map/0 ● Established in 2010 ● Service area ~5,000 km2 ● Population >1.6 million ● ~12,000 staff ● Annual budget $3 billion ● Largest council in Oceania Optimizing Bulk Data Updates in ArcGIS Online
  • 12. Auckland Council (GIS) ● ArcGIS-based public maps & apps ● Datasets are ○ diverse ○ complex ○ large ○ Dynamic ● Need for scalability and resilience (e.g. 2023 Floods) Optimizing Bulk Data Updates in ArcGIS Online
  • 13. Auckland Anniversary Weekend Floods Newsroo m Richard Easther/Twitter NZ Herald Optimizing Bulk Data Updates in ArcGIS Online
  • 15. First Steps No infrastructure maintenance Scalability, Resilience, Security built-in Upgrades taken care of Duration service outage Timing after-hours work Dependent on on-premise data processes Optimizing Bulk Data Updates in ArcGIS Online Host data and apps in ArcGIS Online Challenges around data updates
  • 16. Hosted view ● Requires identical FGDBs (schema) ● Manually update FDGBs and services ● Manually switch view source Optimizing Bulk Data Updates in ArcGIS Online
  • 17. Optimizing Bulk Data Updates in ArcGIS Online
  • 18. Hosted view ● Requires identical FGDBs (schema) ● Manually update FDGBs and services ● Manually switch view source Optimizing Bulk Data Updates in ArcGIS Online
  • 20. What to do? ● ArcGIS Admin REST API ● Esri documentation ● Excel-pattern ● Trial & Error Optimizing Bulk Data Updates in ArcGIS Online
  • 21. The result – FGDB replacement ● 61 transformers ● 7 x HttpCaller ● JsonX-Former ● Dedicated error stream Optimizing Bulk Data Updates in ArcGIS Online
  • 22. The result – data source swap ● 67 transformers ● 8 x HttpCaller ● JsonX-Former ● Dedicated error stream Optimizing Bulk Data Updates in ArcGIS Online
  • 23. The result – master workspace Optimizing Bulk Data Updates in ArcGIS Online
  • 25. 1. Data updates fully automated 2. Downtime reduced to seconds 3. Improved public service delivery Recap Conclusion
  • 29. Agenda 1. Species of National Significance 2. Case Study – Raster Gridding 3. Vector to Raster conversion 4. ESRI vs FME raster gridding tools 5. ArcPy within FME – featureclass processing 6. Tips & Tricks
  • 30.
  • 31. Biodiversity Changes to the landscape and native habitat as a result of human activity have put many species at risk of extinction. Ecological communities are unique and naturally occurring groups of plants and animals. Their presence can be determined by factors such as soil type, position in the landscape, climate and water availability. Environment Protection and Biodiversity Act 1999 (EPBC Act) protects Australia's native species and ecological communities. • Identification and listing of species and ecological communities as threatened • Development of conservation advice and recovery plans for listed species and ecological communities • Register of critical habitat • Recognition of key threatening processes • Where appropriate, reducing the impacts of these processes through threat abatement plans and non-statutory threat abatement advices Model build ArcPy - FME
  • 32. Threatened Ecological Communities of Australia (dcceew.gov.au)
  • 34. Species of National Significance Information as listed in the Environment Protection and Biodiversity Act 1999 (EPBC act) 1. Ecological community name 2. Ecological threatened status ○ Critically Endangered, Endangered, Vulnerable or Conservation Dependent 3. Indicative occurrence (known and predicted areas) produced by spatial ecologists 4. Cell size for public access ○ 1 km grid resolution (0.01°) or ○ ~10km for species classed as (SNES) sensitive by respective States and Territories. 5. Links to further information in the Species Profile and Threats Database (SPRAT) Model build ArcPy - FME
  • 35. Why Generalize? Data source: Australia - Species of National Environmental Significance Distributions (public grids) | Find Environmental Data (dcceew.gov.au) To provide information to the community about Australia’s protected species, and in line with the Government’s policy of open data access, the Department’s threatened and migratory species distributions have been generalized to 1km and 10km The generalized product is aimed at addressing concerns regarding the release of detailed locations of species sensitive to illegal collection and disturbance while still providing public access to the distributions of threatened species. Indicative occurrence - coded to indicate species presence (pres_rank) with: 1 ‘Species or species habitat likely to occur’ and 2 ‘Species or species habitat may occur’. Model build ArcPy - FME
  • 36. Processing Species data Department was seeking to improve spatial generalization methods to process Species Distributions ● Current automated python scripts are difficult to maintain. ● Manual steps and QA checks run several times per week to keep it up to date ● Complex set of business rules dictate which species are to use private or public grids… ● Hiding details for sensitive species Can an FME workflow replace or support the above requirements? Model build ArcPy - FME
  • 37. Model build ArcPy - FME Species Distribution Polygons SPRAT changes (insert, update, delete) Public 1km Public 10km CHANGEDATE WORK_PERFORMED FME_DB_OPERATION 20220811 Deyeuxia ramosa (87970) - SEAP new distribution INSERT 20220811 Nematolepis rhytidophylla (64936) - SEAP updated distribution INSERT 20220811 Pherosphaera fitzgeraldii (40324) - SEAP updated distribution UPDATE 20220811 Syncomistes rastellus (88733) - SEAP new distribution INSERT (filter) Individual Taxon cell assignment type = maximum area pres_rank = gridcode priority = 100-gridcode cell_size = 0.01, 0.1 Generalisation 1. Vector to Raster 2. Raster to Vector snap raster (assign) Sensitive Species 46 36 36 36 36 26 Original Polygon 26 26 1km Raster Grid 10km Raster Grid Generalised Polygon
  • 38. ESRI Gridding Existing approach…. Polygon to raster conversion • Cell assignment Maximum Area • Priority is specified (100-Presence_Rank) e.g., 100 minus 26 = 74. The small “Known” presence given largest cell yields due to high 74 priority • Environment Snap raster used as a template raster grid (next slide) 64 64 64 64 74 54 64 74 54 54 74 64 64 64 64 Model build ArcPy - FME
  • 39. Snap raster reference An environment used to snap or align an extent during execution Snapping usually results in a larger output extent than the given extent. Data source: https://pro.arcgis.com/en/pro-app/latest/tool-reference/environment-setti ngs/how-snap-raster-environment-works.htm Figure (b) shows the snapped extent after execution. Figure (a) shows the extent to be snapped. 74 Model build ArcPy - FME
  • 40. FME Gridding Test alternative approaches…. Raster gridding transformers • NumericRasteriser - creates single-band numeric raster. Precede with 3DForcer for Priority and followed by RasterMosaicker where cell assignment (Overlapping Values) = maximum Summary: • use RasterPropertyExtractor after reading snapraster dataset • 1-1 featuremerger with features to process • NumRaster to use values @round(<ground extents>) and cell spacing • slightly slower than ArcGIS • grid extents end up larger • a dissolve is required as secondary step for cell assignment Model build ArcPy - FME
  • 43. ArcPRO - Export to Python Both model builder and the Geoprocessing “History” have an option to export Python script Data Source: https://pro.arcgis.com/en/pro-app/latest/help/analysis/geoprocessing/modelbuilder/exporting-a-model-to-python.htm Model build ArcPy - FME
  • 44. Feature Class Processing - ArcPy in FME 1. Expose _dataset or use a PATH reader 2. AttributeCreator for names of feature classes and file paths for input & output feature classes 3. Run the ArcPy operation on incoming feature class using the PythonCaller transformer 4. Read the results back into the FME workspace with the FeatureReader transformer Model build ArcPy - FME
  • 45. Cell size '0.1', '0.01', '0.001’ equates to 10km, 1km, 100m Summary Vector-Raster Raster to Vector Input feature
  • 46. Prerequisites: ArcPRO license and spatial analyst extension Set workspace - _dataset is from working_gdb FeatureReader Vector to raster – maximum cell, snapraster, priority Raster to vector – using output paths
  • 47. FME Tips using Python • Set environments to working geodatabase + overwrite output true • Variables for workspace cell size = feature.getAttribute(‘_path%’) arcpy.env.snapraster cell_size • Replace(‘’, ‘’) if any spaces in paths • FMElogfile() concatenate attributes in a notification string and inform geoprocessing progress in the transaction log. • Feature cache mode needs to be emptied To avoid locks, errors 160001-170000 table exists, click the overwrite geodatabase in full Data Source: https://community.safe.com/s/article/using-arcpy-for-fme-feature-processing Model build ArcPy - FME
  • 49. FME workflow with ArcGeoprocessing OUTPUTS fileGDB SNES Public Species with metadata SDE SNES Public Species QA report CHILD WORKSPACE INPUTS Workspace Runner metadata Last load date SPRAT Find all the additions/updates to TaxonIDs Generalising ArcPy – Polygon-Raster-Polygon generaliser_working.gdb & snap raster Process individual Taxons Species Grids Business rules Sensitive Species Resolution Synonymy wkhtmltopdf PARENT WORKSPACE Model build ArcPy - FME
  • 50. Summary • ArcGIS gridding functionality & approach were most consistent to the generalization requirements for processing species distribution grids. • FME superior in terms of orchestrating the integration visually. Where FME transformers lack ESRI functionality, incorporate ArcPy to do the geoprocessing mid workflow. • Get out and support local conservation efforts! Model build ArcPy - FME
  • 51. Resources • DCCEEW www.dcceew.gov.au/environment/biodiversity/threatened • https://community.safe.com/s/article/notes-on-fme-and-esri-versions-and-compatibility • ESRI How Polygon To Raster works—ArcGIS Pro • ESRI ArcPro snap raster • FME NumericRasteriser Popularity = 167/468 • Using Arcpy for FME Feature Processing Model build ArcPy - FME
  • 54. Get our Ebook Spatial Data for the Enterprise fme.ly/gzc Guided learning experiences at your fingertips community.safe.com /s/academy FME Academy Resources & Next Steps
  • 55. Check out how-to’s & demos in the knowledge base community.safe.com /s/knowledge-base Knowledge Base Webinars Upcoming & on-demand webinars safe.com/webinars Resources & Next Steps
  • 56. We’d love to help you get started. Get in touch with us at info@safe.com Experience the FME Accelerator Contact Us Unlock the power of your data in only 90 minutes Register for free at fme.safe.com/accelerator Resources & Next Steps
  • 57. ClaimYour Community Badge & Dive into the new Community! ● Get community badges for watching webinars ● community.safe.com ● Today’s code: APWFG Join the Community today!
  • 58. 6 Q&A
  • 59. ThankYou Recap of Next Steps 1 Join the FME Community 2 Contact James: james.botterill@1spatial.com & Jan: jan.roggisch@locus.co.nz 3 Experience the FME Accelerator Please fill out our webinar survey