SlideShare a Scribd company logo
1 of 65
Download to read offline
The Zero-ETL Approach:
Enhancing Data Agility
and Insights
Christian
Berger
Solutions Specialist
Stewart
Harper
VP of Product
Development
Don
Murray
Co-Founder
Meet our Team
Welcome to Livestorm.
A few ways to engage with us during the webinar:
Audio issues? Click this for 4 simple
troubleshooting steps.
How to download slides
1. Hover over the
slide deck in the
webinar room
2. Click this button
Agenda
1 What is ETL, and Reverse ETL?
2 What about Spatial Data and ETL?
3 What is Zero-ETL?
4 Real Time Data Services and the last data mile
5 Connecting Applications in Real-time
6 High Velocity Data Streams
7 Coming Soon: Data Virtualization
8 Conclusion, Resources, Next Steps, QA
Agenda
Enterprise Integration.
Getting data to where it adds the most
value, when it is needed.
Extract, Transform, and Load (ETL) is one way to
describe the technology to do this.
1
What is ETL, and
Reverse ETL?
On-Premises
ETL = Extract data from
systems, Transform, and
Load into Data stores.
ETL is critical for businesses, combining data
from multiple sources, to provide a consistent
and unified view for reporting and analysis.
On-Premises
ETL = Extract data from
systems, Transform, and
Load into Data stores.
Reverse ETL = Extract data
from data stores, Transform, and
Load into systems.
Reverse ETL operationalizes insights, making
the data actionable in workflows and tools that
support customers, manage operations, and drive
business growth.
Any type of ETL is
● getting and combining data,
● from where it is,
● to where you need it,
● when you need it,
● how you need it!
This data movement is what the
FME Platform is all about.
2
What about
Spatial Data
and ETL?
The world we live in is
Spatial.
How many ways have you used spatial today without even
thinking about it?
Yet most businesses don’t use it to gain new insights!
Spatial Data is critical to get from
digital optimization to digital transformation.
● Spatial data reveals new relationships and insights to make
better decisions.
● Without spatial data you miss opportunities, allocate
resources poorly, and fall behind your competitors.
● Spatial Data is critical for all types of ETL and data
movement.
Unrivalled Data Type support
Spatial Data is
Everywhere
Support for
Spatial Data
Competitor
3
What is
Zero-ETL?
On-Premises
Zero-ETL
● Direct Access to
current data and
services.
● Real-time
Synchronization of
data across systems.
● Flexibility and Agility:
Data flows across all
systems and data
stores in Real-Time.
Real-Time Data
Movement
Zero-ETL delivers real-time data management
and analysis so organizations can respond
quickly to needs and opportunities.
One platform, two technologies
FME Form FME Flow
Data Movement and transformations
(“ETL”) workflows are built here.
Brings life to FME Form workflows
FME Flow Hosted
Safe Software managed FME Flow
fme.safe.com/platform
FME Enterprise Integration Platform
Safe & FME
FME Flow delivers Zero-ETL
4
Real-Time data
services and the last
data mile
Poll or Chat Storm
Question Goes Here.
This will help grab people’s attention and get everyone engaged
right away.
1 2 3
Unlimited, Managed
Users
- User/Role permissions
- Authentication Services
(AD, SAML)
Flexible Engine
Deployment
- Deploy Engines Anywhere
- CPU-Based Pricing
- Remote Engines
Advanced Engine
Management
- Queues & Job Routing
- Active Periods
- Engine Assignment
Data Streaming Workspace Flow Apps
User uploads file
to FME Flow
Workspace App
Data Streaming Workspace Flow Apps
User uploads file
to FME Flow
Workspace App
FME Flow Engines
DataValidation.fmw
executes on next
available engine
Invalid Feature
HTML Report
Submission ID
and Timestamp
Data Streaming Workspace Flow Apps
User uploads file
to FME Flow
Workspace App
Job Queue
DataValidation.fmw
routed to a specified
Queue
Invalid Feature
HTML Report
Submission ID
and Timestamp
Data Streaming Workspace Flow Apps
Invalid Feature
HTML Report
Submission ID
and Timestamp
Data Download Workspace Flow Apps
Job Queue
DataDownload.fmw
routed to a specified
Queue
User Selects
Area of Interest
to Retrieve
Feature(s) from
Job output
provided to client
as downloadable
.zip file
Demo
Customer Story
City of Henderson
Project
Digital Plan Submission
Solution
FME provides self-serve CAD data upload, validation, and
integration into GIS.
Results
● Reduced data integration time by 75%.
● Eliminated the risk of manual error.
● Removed duplicate work.
● Faster processing of submissions.
● Ensured CAD data validation.
“We chose FME for this project because the interface is easy to use and
allows us to provide an adaptable solution to enhance interaction with the
City’s partners.”
- Erika Provost, City of Henderson
5
Connecting
Applications in
Real-Time
FME Flow Automations
Connect different systems so actions in one system update another
● Powerful trigger mechanism via webhooks, data arrival, database
triggers, etc.
● Build multi-step complex workflows
● Take advantage of true parallelism.
● Full orchestration control via split merge blocks.
● Reduce errors and save time by replacing manual tasks with
automation.
Uni-Directional
Integration Architecture
Linear data flow used to repair, validate, and replicate data
from system A to system B
● Triggers can vary from files being received to schedules
and webhooks.
Dynamic data flow(s) used to integrate multiple systems by
synchronizing changes between 2 (or more) systems.
● Typically driven by an action being performed in either
system for near real time processing via webhooks or
topics/notifications.
Bi | Multi-Directional
SFTP
Dropbox
System Event
Azure Event
Grid
JMS
UDP
HTTP Request WebSocket
FTP
Logger
SQS/SNS SMTP
SQS/SNS
Filter
Merge
S3 Bucket
Network
Triggers and
Notifications
FME Flow
Automation Triggered
(Webhook Received
from Survey123)
User submits Survey
Integrate AGOL and Cityworks via REST API
Published FME Workspace (.fmw)
FME Flow
Automation Triggered
Record inserted, updated, or deleted
Record inserted, updated, or deleted
Published FME Workspace executes and synchronizes
changes to between ArcGIS Online Features & Cityworks Work Orders
FME Flow
Automation Triggered
Record inserted, updated, or deleted Record inserted, updated, or deleted
Published FME Workspace executes and synchronizes
changes to features from Cityworks, Salesforce, and
ArcGIS Online
Demo
Customer Story
City of Naperville
Project
Automating time-consuming processes for their Electric,
Public Works, Finance, and Water departments.
Solution
Integrating FME with Cityworks to automate service requests
and work orders.
Results
● 3300+ service requests and work orders automated,
saving hours of manual effort.
● Three staff with specialized coding skills free to focus
on high-value tasks rather than manual work
● Increased collaboration and stronger relationships
between departments
“With FME, we were suddenly saying “yes!” to other departments and
implementing requests quickly. Now, we’re viewed differently in our
interactions. FME makes it easy for us to improve our relationships with
other departments by helping improve their business processes.”
- Danielle DuBose, Database Developer, City of Naperville
6
High Velocity Data
Streams
Let’s start with the data
● Bounded data is finite and has a discrete beginning and end. It is associated with batch processing.
● Unbounded data is infinite, having no discrete beginning or end. It is associated with stream processing.
● FME handles both!
Use Cases
1. Business Applications - Essential for driving real-time business intelligence and process automation.
e.g. Customer orders, airline reservations, insurance claims, bank transactions, telco call detail records (CDR).
2. Digital Information - Captures the dynamics of online engagement and system health.
e.g. clickstreams, logs, market data, email.
3. Internet of Things (IoT) - Capture the vast and growing network of connected devices, translating real-time data from
the physical world into actionable intelligence.
e.g RFID, telemetry SCADA, GPS sensors, smart meters, smart city infrastructure.
Supported IoT &
Messaging Protocols
Simple Streaming Pattern: Filter by Attribute
Reduce data volumes by saving only what you are interested in.
FME Flow Streams
Process data continuously with low latency.
7
Coming Soon:
Data
Virtualization
Data Virtualization
Create Open API compliant APIs on
top of all FME Data Types.
Real-time data (Zero-ETL) to deliver
direct data access via web API.
Effectively making any data cloud
native.
Data Virtualization Value
Agility
● Unified view isolates applications from underlying systems.
Cost Savings
● Simplify infrastructure. Save time and resources on data integration.
Improved Decision Making
● Real-time access to enterprise applications used by decision makers.
Enhanced Data Security and Compliance
● Data access via a secure API improves data security and governance.
Increased Productivity
● Frees IT to focus on strategic projects; business applications access data directly.
Data Virtualization
Connects
Anything
Caching.
We don’t want to run any job unless
needed.
Security.
controlling who and type of access
they have.
Monitoring
Who is using what how much. Rate
limiting, etc.
FME Flow
FME Flow 2024.x
FME Form Workbench - Data Virtualization
8
Conclusion
29+
27K+
128
190
20K+
years of solving data
challenges
FME Community
members
countries with
FME customers
organizations worldwide
global partners with
FME services
30+
29K+
128
140+
25K+
years of solving data
challenges
FME Community
members
countries with
FME customers
organizations worldwide
global partners with
FME services
200K+
users worldwide
Safe & FME
Resources
Get our Ebook
Spatial Data for the
Enterprise
fme.ly/gzc
Guided learning
experiences at your
fingertips
academy.safe.com
FME Academy
Resources
Check out how-to’s &
demos in the knowledge
base
community.safe.com
/s/knowledge-base
Knowledge Base Webinars
Upcoming &
on-demand webinars
safe.com/webinars
Next Steps
We’d love to help you get
started.
Get in touch with us at
info@safe.com
Experience the
FME Accelerator
Contact Us
A world where data is not just a
commodity but a catalyst for
real change.
fme.safe.com/accelerator
Next Steps
ClaimYour Community Badge &
Dive into the new Community!
● Get community badges for watching
webinars
● fme.ly/RewardCodes
● Today’s code:
Join the Community today!
Q&A
ThankYou
Recap of Next Steps
1 Join the FME Community
2 Contact us
3 Experience the FME Accelerator
Please fill out our
webinar survey

More Related Content

Similar to The Zero-ETL Approach: Enhancing Data Agility and Insight

integrating-on-premise-apps-cloud-300329.pdf
integrating-on-premise-apps-cloud-300329.pdfintegrating-on-premise-apps-cloud-300329.pdf
integrating-on-premise-apps-cloud-300329.pdfssusera9d7fc1
 
7 FME Server Use Cases To Convince Your Boss
7 FME Server Use Cases To Convince Your Boss7 FME Server Use Cases To Convince Your Boss
7 FME Server Use Cases To Convince Your BossSafe Software
 
A Winning Strategy for the Digital Economy
A Winning Strategy for the Digital EconomyA Winning Strategy for the Digital Economy
A Winning Strategy for the Digital EconomyEric Kavanagh
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationDenodo
 
Mastering Information Technology During Business Transformation
Mastering Information Technology During Business TransformationMastering Information Technology During Business Transformation
Mastering Information Technology During Business TransformationNetApp
 
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Denodo
 
Power Up Your BI with Geospatial Data
Power Up Your BI with Geospatial Data Power Up Your BI with Geospatial Data
Power Up Your BI with Geospatial Data Safe Software
 
The Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reductionThe Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reductionMongoDB
 
Modernize and Simplify IT Operations Management for DevOps Success
Modernize and Simplify IT Operations Management for DevOps SuccessModernize and Simplify IT Operations Management for DevOps Success
Modernize and Simplify IT Operations Management for DevOps SuccessDevOps.com
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
8.17.11 big data and hadoop with informatica slideshare
8.17.11 big data and hadoop with informatica slideshare8.17.11 big data and hadoop with informatica slideshare
8.17.11 big data and hadoop with informatica slideshareJulianna DeLua
 
Brighttalk understanding the promise of sde - final
Brighttalk   understanding the promise of sde - finalBrighttalk   understanding the promise of sde - final
Brighttalk understanding the promise of sde - finalAndrew White
 
Real time data integration best practices and architecture
Real time data integration best practices and architectureReal time data integration best practices and architecture
Real time data integration best practices and architectureBui Kiet
 
10 Best Data Integration Software Platforms.pdf
10 Best Data Integration Software Platforms.pdf10 Best Data Integration Software Platforms.pdf
10 Best Data Integration Software Platforms.pdfXoxoday Compass
 
Data Virtualization Journey: How to Grow from Single Project and to Enterpris...
Data Virtualization Journey: How to Grow from Single Project and to Enterpris...Data Virtualization Journey: How to Grow from Single Project and to Enterpris...
Data Virtualization Journey: How to Grow from Single Project and to Enterpris...Denodo
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationDenodo
 
Dreamforce 2010: Sales Cloud Integration: Accelerate CRM Adoption and ROI
Dreamforce 2010: Sales Cloud Integration: Accelerate CRM Adoption and ROIDreamforce 2010: Sales Cloud Integration: Accelerate CRM Adoption and ROI
Dreamforce 2010: Sales Cloud Integration: Accelerate CRM Adoption and ROIDarren Cunningham
 
Modernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your DataModernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your DataPrecisely
 
Innovating With Data and Analytics
Innovating With Data and AnalyticsInnovating With Data and Analytics
Innovating With Data and AnalyticsVMware Tanzu
 
RTI Data-Distribution Service (DDS) Master Class 2011
RTI Data-Distribution Service (DDS) Master Class 2011RTI Data-Distribution Service (DDS) Master Class 2011
RTI Data-Distribution Service (DDS) Master Class 2011Gerardo Pardo-Castellote
 

Similar to The Zero-ETL Approach: Enhancing Data Agility and Insight (20)

integrating-on-premise-apps-cloud-300329.pdf
integrating-on-premise-apps-cloud-300329.pdfintegrating-on-premise-apps-cloud-300329.pdf
integrating-on-premise-apps-cloud-300329.pdf
 
7 FME Server Use Cases To Convince Your Boss
7 FME Server Use Cases To Convince Your Boss7 FME Server Use Cases To Convince Your Boss
7 FME Server Use Cases To Convince Your Boss
 
A Winning Strategy for the Digital Economy
A Winning Strategy for the Digital EconomyA Winning Strategy for the Digital Economy
A Winning Strategy for the Digital Economy
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and Visualization
 
Mastering Information Technology During Business Transformation
Mastering Information Technology During Business TransformationMastering Information Technology During Business Transformation
Mastering Information Technology During Business Transformation
 
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
 
Power Up Your BI with Geospatial Data
Power Up Your BI with Geospatial Data Power Up Your BI with Geospatial Data
Power Up Your BI with Geospatial Data
 
The Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reductionThe Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reduction
 
Modernize and Simplify IT Operations Management for DevOps Success
Modernize and Simplify IT Operations Management for DevOps SuccessModernize and Simplify IT Operations Management for DevOps Success
Modernize and Simplify IT Operations Management for DevOps Success
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
8.17.11 big data and hadoop with informatica slideshare
8.17.11 big data and hadoop with informatica slideshare8.17.11 big data and hadoop with informatica slideshare
8.17.11 big data and hadoop with informatica slideshare
 
Brighttalk understanding the promise of sde - final
Brighttalk   understanding the promise of sde - finalBrighttalk   understanding the promise of sde - final
Brighttalk understanding the promise of sde - final
 
Real time data integration best practices and architecture
Real time data integration best practices and architectureReal time data integration best practices and architecture
Real time data integration best practices and architecture
 
10 Best Data Integration Software Platforms.pdf
10 Best Data Integration Software Platforms.pdf10 Best Data Integration Software Platforms.pdf
10 Best Data Integration Software Platforms.pdf
 
Data Virtualization Journey: How to Grow from Single Project and to Enterpris...
Data Virtualization Journey: How to Grow from Single Project and to Enterpris...Data Virtualization Journey: How to Grow from Single Project and to Enterpris...
Data Virtualization Journey: How to Grow from Single Project and to Enterpris...
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and Visualization
 
Dreamforce 2010: Sales Cloud Integration: Accelerate CRM Adoption and ROI
Dreamforce 2010: Sales Cloud Integration: Accelerate CRM Adoption and ROIDreamforce 2010: Sales Cloud Integration: Accelerate CRM Adoption and ROI
Dreamforce 2010: Sales Cloud Integration: Accelerate CRM Adoption and ROI
 
Modernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your DataModernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your Data
 
Innovating With Data and Analytics
Innovating With Data and AnalyticsInnovating With Data and Analytics
Innovating With Data and Analytics
 
RTI Data-Distribution Service (DDS) Master Class 2011
RTI Data-Distribution Service (DDS) Master Class 2011RTI Data-Distribution Service (DDS) Master Class 2011
RTI Data-Distribution Service (DDS) Master Class 2011
 

More from Safe Software

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemSafe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISSafe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersSafe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsSafe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategySafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Safe Software
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMESafe Software
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Safe Software
 
Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Safe Software
 
Bringing Open Data Integration to the (SWECO) People
Bringing Open Data Integration to the (SWECO) PeopleBringing Open Data Integration to the (SWECO) People
Bringing Open Data Integration to the (SWECO) PeopleSafe Software
 
Democratizing Transformer Creation in FME
Democratizing Transformer Creation in FMEDemocratizing Transformer Creation in FME
Democratizing Transformer Creation in FMESafe Software
 

More from Safe Software (20)

Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FME
 
Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...Identification of landscape features eligible for EU subsidy - Analysis of la...
Identification of landscape features eligible for EU subsidy - Analysis of la...
 
Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework Utilizing FME as an API Test Framework
Utilizing FME as an API Test Framework
 
Bringing Open Data Integration to the (SWECO) People
Bringing Open Data Integration to the (SWECO) PeopleBringing Open Data Integration to the (SWECO) People
Bringing Open Data Integration to the (SWECO) People
 
Democratizing Transformer Creation in FME
Democratizing Transformer Creation in FMEDemocratizing Transformer Creation in FME
Democratizing Transformer Creation in FME
 

Recently uploaded

Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 

Recently uploaded (20)

Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort ServiceHot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 

The Zero-ETL Approach: Enhancing Data Agility and Insight

  • 1. The Zero-ETL Approach: Enhancing Data Agility and Insights
  • 2. Christian Berger Solutions Specialist Stewart Harper VP of Product Development Don Murray Co-Founder Meet our Team
  • 3. Welcome to Livestorm. A few ways to engage with us during the webinar: Audio issues? Click this for 4 simple troubleshooting steps.
  • 4. How to download slides 1. Hover over the slide deck in the webinar room 2. Click this button
  • 5. Agenda 1 What is ETL, and Reverse ETL? 2 What about Spatial Data and ETL? 3 What is Zero-ETL? 4 Real Time Data Services and the last data mile 5 Connecting Applications in Real-time 6 High Velocity Data Streams 7 Coming Soon: Data Virtualization 8 Conclusion, Resources, Next Steps, QA Agenda
  • 6. Enterprise Integration. Getting data to where it adds the most value, when it is needed. Extract, Transform, and Load (ETL) is one way to describe the technology to do this.
  • 7. 1 What is ETL, and Reverse ETL?
  • 8. On-Premises ETL = Extract data from systems, Transform, and Load into Data stores.
  • 9. ETL is critical for businesses, combining data from multiple sources, to provide a consistent and unified view for reporting and analysis.
  • 10. On-Premises ETL = Extract data from systems, Transform, and Load into Data stores. Reverse ETL = Extract data from data stores, Transform, and Load into systems.
  • 11. Reverse ETL operationalizes insights, making the data actionable in workflows and tools that support customers, manage operations, and drive business growth.
  • 12. Any type of ETL is ● getting and combining data, ● from where it is, ● to where you need it, ● when you need it, ● how you need it! This data movement is what the FME Platform is all about.
  • 14. The world we live in is Spatial. How many ways have you used spatial today without even thinking about it? Yet most businesses don’t use it to gain new insights!
  • 15. Spatial Data is critical to get from digital optimization to digital transformation. ● Spatial data reveals new relationships and insights to make better decisions. ● Without spatial data you miss opportunities, allocate resources poorly, and fall behind your competitors. ● Spatial Data is critical for all types of ETL and data movement.
  • 16. Unrivalled Data Type support Spatial Data is Everywhere
  • 19. On-Premises Zero-ETL ● Direct Access to current data and services. ● Real-time Synchronization of data across systems. ● Flexibility and Agility: Data flows across all systems and data stores in Real-Time. Real-Time Data Movement
  • 20. Zero-ETL delivers real-time data management and analysis so organizations can respond quickly to needs and opportunities.
  • 21. One platform, two technologies FME Form FME Flow Data Movement and transformations (“ETL”) workflows are built here. Brings life to FME Form workflows FME Flow Hosted Safe Software managed FME Flow fme.safe.com/platform FME Enterprise Integration Platform Safe & FME
  • 22. FME Flow delivers Zero-ETL
  • 23. 4 Real-Time data services and the last data mile
  • 24. Poll or Chat Storm Question Goes Here. This will help grab people’s attention and get everyone engaged right away.
  • 25. 1 2 3 Unlimited, Managed Users - User/Role permissions - Authentication Services (AD, SAML) Flexible Engine Deployment - Deploy Engines Anywhere - CPU-Based Pricing - Remote Engines Advanced Engine Management - Queues & Job Routing - Active Periods - Engine Assignment
  • 26. Data Streaming Workspace Flow Apps User uploads file to FME Flow Workspace App
  • 27. Data Streaming Workspace Flow Apps User uploads file to FME Flow Workspace App FME Flow Engines DataValidation.fmw executes on next available engine Invalid Feature HTML Report Submission ID and Timestamp
  • 28. Data Streaming Workspace Flow Apps User uploads file to FME Flow Workspace App Job Queue DataValidation.fmw routed to a specified Queue Invalid Feature HTML Report Submission ID and Timestamp
  • 29. Data Streaming Workspace Flow Apps Invalid Feature HTML Report Submission ID and Timestamp
  • 30.
  • 31. Data Download Workspace Flow Apps Job Queue DataDownload.fmw routed to a specified Queue User Selects Area of Interest to Retrieve Feature(s) from Job output provided to client as downloadable .zip file
  • 32. Demo
  • 33.
  • 34. Customer Story City of Henderson Project Digital Plan Submission Solution FME provides self-serve CAD data upload, validation, and integration into GIS. Results ● Reduced data integration time by 75%. ● Eliminated the risk of manual error. ● Removed duplicate work. ● Faster processing of submissions. ● Ensured CAD data validation. “We chose FME for this project because the interface is easy to use and allows us to provide an adaptable solution to enhance interaction with the City’s partners.” - Erika Provost, City of Henderson
  • 36. FME Flow Automations Connect different systems so actions in one system update another ● Powerful trigger mechanism via webhooks, data arrival, database triggers, etc. ● Build multi-step complex workflows ● Take advantage of true parallelism. ● Full orchestration control via split merge blocks. ● Reduce errors and save time by replacing manual tasks with automation.
  • 37. Uni-Directional Integration Architecture Linear data flow used to repair, validate, and replicate data from system A to system B ● Triggers can vary from files being received to schedules and webhooks. Dynamic data flow(s) used to integrate multiple systems by synchronizing changes between 2 (or more) systems. ● Typically driven by an action being performed in either system for near real time processing via webhooks or topics/notifications. Bi | Multi-Directional
  • 38. SFTP Dropbox System Event Azure Event Grid JMS UDP HTTP Request WebSocket FTP Logger SQS/SNS SMTP SQS/SNS Filter Merge S3 Bucket Network Triggers and Notifications
  • 39. FME Flow Automation Triggered (Webhook Received from Survey123) User submits Survey Integrate AGOL and Cityworks via REST API Published FME Workspace (.fmw)
  • 40. FME Flow Automation Triggered Record inserted, updated, or deleted Record inserted, updated, or deleted Published FME Workspace executes and synchronizes changes to between ArcGIS Online Features & Cityworks Work Orders
  • 41. FME Flow Automation Triggered Record inserted, updated, or deleted Record inserted, updated, or deleted Published FME Workspace executes and synchronizes changes to features from Cityworks, Salesforce, and ArcGIS Online
  • 42. Demo
  • 43.
  • 44. Customer Story City of Naperville Project Automating time-consuming processes for their Electric, Public Works, Finance, and Water departments. Solution Integrating FME with Cityworks to automate service requests and work orders. Results ● 3300+ service requests and work orders automated, saving hours of manual effort. ● Three staff with specialized coding skills free to focus on high-value tasks rather than manual work ● Increased collaboration and stronger relationships between departments “With FME, we were suddenly saying “yes!” to other departments and implementing requests quickly. Now, we’re viewed differently in our interactions. FME makes it easy for us to improve our relationships with other departments by helping improve their business processes.” - Danielle DuBose, Database Developer, City of Naperville
  • 46. Let’s start with the data ● Bounded data is finite and has a discrete beginning and end. It is associated with batch processing. ● Unbounded data is infinite, having no discrete beginning or end. It is associated with stream processing. ● FME handles both!
  • 47. Use Cases 1. Business Applications - Essential for driving real-time business intelligence and process automation. e.g. Customer orders, airline reservations, insurance claims, bank transactions, telco call detail records (CDR). 2. Digital Information - Captures the dynamics of online engagement and system health. e.g. clickstreams, logs, market data, email. 3. Internet of Things (IoT) - Capture the vast and growing network of connected devices, translating real-time data from the physical world into actionable intelligence. e.g RFID, telemetry SCADA, GPS sensors, smart meters, smart city infrastructure.
  • 49. Simple Streaming Pattern: Filter by Attribute Reduce data volumes by saving only what you are interested in.
  • 50. FME Flow Streams Process data continuously with low latency.
  • 52. Data Virtualization Create Open API compliant APIs on top of all FME Data Types. Real-time data (Zero-ETL) to deliver direct data access via web API. Effectively making any data cloud native.
  • 53. Data Virtualization Value Agility ● Unified view isolates applications from underlying systems. Cost Savings ● Simplify infrastructure. Save time and resources on data integration. Improved Decision Making ● Real-time access to enterprise applications used by decision makers. Enhanced Data Security and Compliance ● Data access via a secure API improves data security and governance. Increased Productivity ● Frees IT to focus on strategic projects; business applications access data directly.
  • 54. Data Virtualization Connects Anything Caching. We don’t want to run any job unless needed. Security. controlling who and type of access they have. Monitoring Who is using what how much. Rate limiting, etc. FME Flow
  • 56. FME Form Workbench - Data Virtualization
  • 58. 29+ 27K+ 128 190 20K+ years of solving data challenges FME Community members countries with FME customers organizations worldwide global partners with FME services 30+ 29K+ 128 140+ 25K+ years of solving data challenges FME Community members countries with FME customers organizations worldwide global partners with FME services 200K+ users worldwide Safe & FME
  • 60. Get our Ebook Spatial Data for the Enterprise fme.ly/gzc Guided learning experiences at your fingertips academy.safe.com FME Academy Resources Check out how-to’s & demos in the knowledge base community.safe.com /s/knowledge-base Knowledge Base Webinars Upcoming & on-demand webinars safe.com/webinars
  • 62. We’d love to help you get started. Get in touch with us at info@safe.com Experience the FME Accelerator Contact Us A world where data is not just a commodity but a catalyst for real change. fme.safe.com/accelerator Next Steps
  • 63. ClaimYour Community Badge & Dive into the new Community! ● Get community badges for watching webinars ● fme.ly/RewardCodes ● Today’s code: Join the Community today!
  • 64. Q&A
  • 65. ThankYou Recap of Next Steps 1 Join the FME Community 2 Contact us 3 Experience the FME Accelerator Please fill out our webinar survey