SlideShare a Scribd company logo
The Zero-ETL Approach:
Enhancing Data Agility
and Insights
Christian
Berger
Solutions Specialist
Stewart
Harper
VP of Product
Development
Don
Murray
Co-Founder
Meet our Team
Welcome to Livestorm.
A few ways to engage with us during the webinar:
Audio issues? Click this for 4 simple
troubleshooting steps.
How to download slides
1. Hover over the
slide deck in the
webinar room
2. Click this button
Agenda
1 What is ETL, and Reverse ETL?
2 What about Spatial Data and ETL?
3 What is Zero-ETL?
4 Real Time Data Services and the last data mile
5 Connecting Applications in Real-time
6 High Velocity Data Streams
7 Coming Soon: Data Virtualization
8 Conclusion, Resources, Next Steps, QA
Agenda
Enterprise Integration.
Getting data to where it adds the most
value, when it is needed.
Extract, Transform, and Load (ETL) is one way to
describe the technology to do this.
1
What is ETL, and
Reverse ETL?
On-Premises
ETL = Extract data from
systems, Transform, and
Load into Data stores.
ETL is critical for businesses, combining data
from multiple sources, to provide a consistent
and unified view for reporting and analysis.
On-Premises
ETL = Extract data from
systems, Transform, and
Load into Data stores.
Reverse ETL = Extract data
from data stores, Transform, and
Load into systems.
Reverse ETL operationalizes insights, making
the data actionable in workflows and tools that
support customers, manage operations, and drive
business growth.
Any type of ETL is
● getting and combining data,
● from where it is,
● to where you need it,
● when you need it,
● how you need it!
This data movement is what the
FME Platform is all about.
2
What about
Spatial Data
and ETL?
The world we live in is
Spatial.
How many ways have you used spatial today without even
thinking about it?
Yet most businesses don’t use it to gain new insights!
Spatial Data is critical to get from
digital optimization to digital transformation.
● Spatial data reveals new relationships and insights to make
better decisions.
● Without spatial data you miss opportunities, allocate
resources poorly, and fall behind your competitors.
● Spatial Data is critical for all types of ETL and data
movement.
Unrivalled Data Type support
Spatial Data is
Everywhere
Support for
Spatial Data
Competitor
3
What is
Zero-ETL?
On-Premises
Zero-ETL
● Direct Access to
current data and
services.
● Real-time
Synchronization of
data across systems.
● Flexibility and Agility:
Data flows across all
systems and data
stores in Real-Time.
Real-Time Data
Movement
Zero-ETL delivers real-time data management
and analysis so organizations can respond
quickly to needs and opportunities.
One platform, two technologies
FME Form FME Flow
Data Movement and transformations
(“ETL”) workflows are built here.
Brings life to FME Form workflows
FME Flow Hosted
Safe Software managed FME Flow
fme.safe.com/platform
FME Enterprise Integration Platform
Safe & FME
FME Flow delivers Zero-ETL
4
Real-Time data
services and the last
data mile
1 2 3
Unlimited, Managed
Users
- User/Role permissions
- Authentication Services
(AD, SAML)
Flexible Engine
Deployment
- Deploy Engines Anywhere
- CPU-Based Pricing
- Remote Engines
Advanced Engine
Management
- Queues & Job Routing
- Active Periods
- Engine Assignment
Data Streaming Workspace Flow Apps
User uploads file
to FME Flow
Workspace App
Data Streaming Workspace Flow Apps
User uploads file
to FME Flow
Workspace App
FME Flow Engines
DataValidation.fmw
executes on next
available engine
Invalid Feature
HTML Report
Submission ID
and Timestamp
Data Streaming Workspace Flow Apps
User uploads file
to FME Flow
Workspace App
Job Queue
DataValidation.fmw
routed to a specified
Queue
Invalid Feature
HTML Report
Submission ID
and Timestamp
Data Streaming Workspace Flow Apps
Invalid Feature
HTML Report
Submission ID
and Timestamp
Data Download Workspace Flow Apps
Job Queue
DataDownload.fmw
routed to a specified
Queue
User Selects
Area of Interest
to Retrieve
Feature(s) from
Job output
provided to client
as downloadable
.zip file
Customer Story
City of Henderson
Project
Digital Plan Submission
Solution
FME provides self-serve CAD data upload, validation, and
integration into GIS.
Results
● Reduced data integration time by 75%.
● Eliminated the risk of manual error.
● Removed duplicate work.
● Faster processing of submissions.
● Ensured CAD data validation.
“We chose FME for this project because the interface is easy to use and
allows us to provide an adaptable solution to enhance interaction with the
City’s partners.”
- Erika Provost, City of Henderson
5
Connecting
Applications in
Real-Time
FME Flow Automations
Connect different systems so actions in one system update another
● Powerful trigger mechanism via webhooks, data arrival, database
triggers, etc.
● Build multi-step complex workflows
● Take advantage of true parallelism.
● Full orchestration control via split merge blocks.
● Reduce errors and save time by replacing manual tasks with
automation.
Uni-Directional
Integration Architecture
Linear data flow used to repair, validate, and replicate data
from system A to system B
● Triggers can vary from files being received to schedules
and webhooks.
Dynamic data flow(s) used to integrate multiple systems by
synchronizing changes between 2 (or more) systems.
● Typically driven by an action being performed in either
system for near real time processing via webhooks or
topics/notifications.
Bi | Multi-Directional
SFTP
Dropbox
System Event
Azure Event
Grid
JMS
UDP
HTTP Request WebSocket
FTP
Logger
SQS/SNS SMTP
SQS/SNS
Filter
Merge
S3 Bucket
Network
Triggers and
Notifications
FME Flow
Automation Triggered
(Webhook Received
from Survey123)
User submits Survey
Integrate AGOL and Cityworks via REST API
Published FME Workspace (.fmw)
FME Flow
Automation Triggered
Record inserted, updated, or deleted
Record inserted, updated, or deleted
Published FME Workspace executes and synchronizes
changes to between ArcGIS Online Features & Cityworks Work Orders
FME Flow
Automation Triggered
Record inserted, updated, or deleted Record inserted, updated, or deleted
Published FME Workspace executes and synchronizes
changes to features from Cityworks, Salesforce, and
ArcGIS Online
Demo
Customer Story
City of Naperville
Project
Automating time-consuming processes for their Electric,
Public Works, Finance, and Water departments.
Solution
Integrating FME with Cityworks to automate service requests
and work orders.
Results
● 3300+ service requests and work orders automated,
saving hours of manual effort.
● Three staff with specialized coding skills free to focus
on high-value tasks rather than manual work
● Increased collaboration and stronger relationships
between departments
“With FME, we were suddenly saying “yes!” to other departments and
implementing requests quickly. Now, we’re viewed differently in our
interactions. FME makes it easy for us to improve our relationships with
other departments by helping improve their business processes.”
- Danielle DuBose, Database Developer, City of Naperville
6
High Velocity Data
Streams
Let’s start with the data
● Bounded data is finite and has a discrete beginning and end. It is associated with batch processing.
● Unbounded data is infinite, having no discrete beginning or end. It is associated with stream processing.
● FME handles both!
Use Cases
1. Business Applications - Essential for driving real-time business intelligence and process automation.
e.g. Customer orders, airline reservations, insurance claims, bank transactions, telco call detail records (CDR).
2. Digital Information - Captures the dynamics of online engagement and system health.
e.g. clickstreams, logs, market data, email.
3. Internet of Things (IoT) - Capture the vast and growing network of connected devices, translating real-time data from
the physical world into actionable intelligence.
e.g RFID, telemetry SCADA, GPS sensors, smart meters, smart city infrastructure.
Supported IoT &
Messaging Protocols
Simple Streaming Pattern: Filter by Attribute
Reduce data volumes by saving only what you are interested in.
FME Flow Streams
Process data continuously with low latency.
7
Coming Soon:
Data
Virtualization
Data Virtualization
Create Open API compliant APIs on
top of all FME Data Types.
Real-time data (Zero-ETL) to deliver
direct data access via web API.
Effectively making any data cloud
native.
Data Virtualization Value
Agility
● Unified view isolates applications from underlying systems.
Cost Savings
● Simplify infrastructure. Save time and resources on data integration.
Improved Decision Making
● Real-time access to enterprise applications used by decision makers.
Enhanced Data Security and Compliance
● Data access via a secure API improves data security and governance.
Increased Productivity
● Frees IT to focus on strategic projects; business applications access data directly.
Data Virtualization
Connects
Anything
Caching.
We don’t want to run any job unless
needed.
Security.
controlling who and type of access
they have.
Monitoring
Who is using what how much. Rate
limiting, etc.
FME Flow
FME Flow 2024.x
FME Form Workbench - Data Virtualization
8
Conclusion
29+
27K+
128
190
20K+
years of solving data
challenges
FME Community
members
countries with
FME customers
organizations worldwide
global partners with
FME services
30+
29K+
128
140+
25K+
years of solving data
challenges
FME Community
members
countries with
FME customers
organizations worldwide
global partners with
FME services
200K+
users worldwide
Safe & FME
Resources
Get our Ebook
Spatial Data for the
Enterprise
fme.ly/gzc
Guided learning
experiences at your
fingertips
academy.safe.com
FME Academy
Resources
Check out how-to’s &
demos in the knowledge
base
community.safe.com
/s/knowledge-base
Knowledge Base Webinars
Upcoming &
on-demand webinars
safe.com/webinars
Next Steps
We’d love to help you get
started.
Get in touch with us at
info@safe.com
Experience the
FME Accelerator
Contact Us
A world where data is not just a
commodity but a catalyst for
real change.
fme.safe.com/accelerator
Next Steps
ClaimYour Community Badge &
Dive into the new Community!
● Get community badges for watching
webinars
● community.safe.com
● Today’s code: C35A62
Join the Community today!
Q&A
ThankYou
Recap of Next Steps
1 Join the FME Community
2 Contact us
3 Experience the FME Accelerator
Please fill out our
webinar survey

More Related Content

Similar to The Zero-ETL Approach: Enhancing Data Agility and Insight

Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Denodo
 
Accelerate Digital Transformation with Data Virtualization in Banking, Financ...
Accelerate Digital Transformation with Data Virtualization in Banking, Financ...Accelerate Digital Transformation with Data Virtualization in Banking, Financ...
Accelerate Digital Transformation with Data Virtualization in Banking, Financ...
Denodo
 
Brighttalk understanding the promise of sde - final
Brighttalk   understanding the promise of sde - finalBrighttalk   understanding the promise of sde - final
Brighttalk understanding the promise of sde - final
Andrew White
 

Similar to The Zero-ETL Approach: Enhancing Data Agility and Insight (20)

EMA Presentation: Driving Business Value with Continuous Operational Intellig...
EMA Presentation: Driving Business Value with Continuous Operational Intellig...EMA Presentation: Driving Business Value with Continuous Operational Intellig...
EMA Presentation: Driving Business Value with Continuous Operational Intellig...
 
Confluent Partner Tech Talk with BearingPoint
Confluent Partner Tech Talk with BearingPointConfluent Partner Tech Talk with BearingPoint
Confluent Partner Tech Talk with BearingPoint
 
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
 
Jazz for Service Management
Jazz for Service ManagementJazz for Service Management
Jazz for Service Management
 
Mastering Information Technology During Business Transformation
Mastering Information Technology During Business TransformationMastering Information Technology During Business Transformation
Mastering Information Technology During Business Transformation
 
A Winning Strategy for the Digital Economy
A Winning Strategy for the Digital EconomyA Winning Strategy for the Digital Economy
A Winning Strategy for the Digital Economy
 
integrating-on-premise-apps-cloud-300329.pdf
integrating-on-premise-apps-cloud-300329.pdfintegrating-on-premise-apps-cloud-300329.pdf
integrating-on-premise-apps-cloud-300329.pdf
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and Visualization
 
Power Up Your BI with Geospatial Data
Power Up Your BI with Geospatial Data Power Up Your BI with Geospatial Data
Power Up Your BI with Geospatial Data
 
The Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reductionThe Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reduction
 
10 Best Data Integration Software Platforms.pdf
10 Best Data Integration Software Platforms.pdf10 Best Data Integration Software Platforms.pdf
10 Best Data Integration Software Platforms.pdf
 
Accelerate Digital Transformation with Data Virtualization in Banking, Financ...
Accelerate Digital Transformation with Data Virtualization in Banking, Financ...Accelerate Digital Transformation with Data Virtualization in Banking, Financ...
Accelerate Digital Transformation with Data Virtualization in Banking, Financ...
 
8.17.11 big data and hadoop with informatica slideshare
8.17.11 big data and hadoop with informatica slideshare8.17.11 big data and hadoop with informatica slideshare
8.17.11 big data and hadoop with informatica slideshare
 
Brighttalk understanding the promise of sde - final
Brighttalk   understanding the promise of sde - finalBrighttalk   understanding the promise of sde - final
Brighttalk understanding the promise of sde - final
 
A Detailed Guide To DataOps
A Detailed Guide To DataOpsA Detailed Guide To DataOps
A Detailed Guide To DataOps
 
Modernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your DataModernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your Data
 
7 FME Server Use Cases To Convince Your Boss
7 FME Server Use Cases To Convince Your Boss7 FME Server Use Cases To Convince Your Boss
7 FME Server Use Cases To Convince Your Boss
 
NTGapps DTB Platform.pdf
NTGapps DTB Platform.pdfNTGapps DTB Platform.pdf
NTGapps DTB Platform.pdf
 
Real time data integration best practices and architecture
Real time data integration best practices and architectureReal time data integration best practices and architecture
Real time data integration best practices and architecture
 
Modernize and Simplify IT Operations Management for DevOps Success
Modernize and Simplify IT Operations Management for DevOps SuccessModernize and Simplify IT Operations Management for DevOps Success
Modernize and Simplify IT Operations Management for DevOps Success
 

More from Safe Software

Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
Safe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Safe Software
 

More from Safe Software (20)

Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FME
 

Recently uploaded

Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
Bhaskar Mitra
 

Recently uploaded (20)

Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
In-Depth Performance Testing Guide for IT Professionals
In-Depth Performance Testing Guide for IT ProfessionalsIn-Depth Performance Testing Guide for IT Professionals
In-Depth Performance Testing Guide for IT Professionals
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
Introduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG EvaluationIntroduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG Evaluation
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 

The Zero-ETL Approach: Enhancing Data Agility and Insight

  • 1. The Zero-ETL Approach: Enhancing Data Agility and Insights
  • 2. Christian Berger Solutions Specialist Stewart Harper VP of Product Development Don Murray Co-Founder Meet our Team
  • 3. Welcome to Livestorm. A few ways to engage with us during the webinar: Audio issues? Click this for 4 simple troubleshooting steps.
  • 4. How to download slides 1. Hover over the slide deck in the webinar room 2. Click this button
  • 5. Agenda 1 What is ETL, and Reverse ETL? 2 What about Spatial Data and ETL? 3 What is Zero-ETL? 4 Real Time Data Services and the last data mile 5 Connecting Applications in Real-time 6 High Velocity Data Streams 7 Coming Soon: Data Virtualization 8 Conclusion, Resources, Next Steps, QA Agenda
  • 6. Enterprise Integration. Getting data to where it adds the most value, when it is needed. Extract, Transform, and Load (ETL) is one way to describe the technology to do this.
  • 7. 1 What is ETL, and Reverse ETL?
  • 8. On-Premises ETL = Extract data from systems, Transform, and Load into Data stores.
  • 9. ETL is critical for businesses, combining data from multiple sources, to provide a consistent and unified view for reporting and analysis.
  • 10. On-Premises ETL = Extract data from systems, Transform, and Load into Data stores. Reverse ETL = Extract data from data stores, Transform, and Load into systems.
  • 11. Reverse ETL operationalizes insights, making the data actionable in workflows and tools that support customers, manage operations, and drive business growth.
  • 12. Any type of ETL is ● getting and combining data, ● from where it is, ● to where you need it, ● when you need it, ● how you need it! This data movement is what the FME Platform is all about.
  • 14. The world we live in is Spatial. How many ways have you used spatial today without even thinking about it? Yet most businesses don’t use it to gain new insights!
  • 15. Spatial Data is critical to get from digital optimization to digital transformation. ● Spatial data reveals new relationships and insights to make better decisions. ● Without spatial data you miss opportunities, allocate resources poorly, and fall behind your competitors. ● Spatial Data is critical for all types of ETL and data movement.
  • 16. Unrivalled Data Type support Spatial Data is Everywhere
  • 19. On-Premises Zero-ETL ● Direct Access to current data and services. ● Real-time Synchronization of data across systems. ● Flexibility and Agility: Data flows across all systems and data stores in Real-Time. Real-Time Data Movement
  • 20. Zero-ETL delivers real-time data management and analysis so organizations can respond quickly to needs and opportunities.
  • 21. One platform, two technologies FME Form FME Flow Data Movement and transformations (“ETL”) workflows are built here. Brings life to FME Form workflows FME Flow Hosted Safe Software managed FME Flow fme.safe.com/platform FME Enterprise Integration Platform Safe & FME
  • 22. FME Flow delivers Zero-ETL
  • 23. 4 Real-Time data services and the last data mile
  • 24. 1 2 3 Unlimited, Managed Users - User/Role permissions - Authentication Services (AD, SAML) Flexible Engine Deployment - Deploy Engines Anywhere - CPU-Based Pricing - Remote Engines Advanced Engine Management - Queues & Job Routing - Active Periods - Engine Assignment
  • 25. Data Streaming Workspace Flow Apps User uploads file to FME Flow Workspace App
  • 26. Data Streaming Workspace Flow Apps User uploads file to FME Flow Workspace App FME Flow Engines DataValidation.fmw executes on next available engine Invalid Feature HTML Report Submission ID and Timestamp
  • 27. Data Streaming Workspace Flow Apps User uploads file to FME Flow Workspace App Job Queue DataValidation.fmw routed to a specified Queue Invalid Feature HTML Report Submission ID and Timestamp
  • 28. Data Streaming Workspace Flow Apps Invalid Feature HTML Report Submission ID and Timestamp
  • 29.
  • 30. Data Download Workspace Flow Apps Job Queue DataDownload.fmw routed to a specified Queue User Selects Area of Interest to Retrieve Feature(s) from Job output provided to client as downloadable .zip file
  • 31.
  • 32. Customer Story City of Henderson Project Digital Plan Submission Solution FME provides self-serve CAD data upload, validation, and integration into GIS. Results ● Reduced data integration time by 75%. ● Eliminated the risk of manual error. ● Removed duplicate work. ● Faster processing of submissions. ● Ensured CAD data validation. “We chose FME for this project because the interface is easy to use and allows us to provide an adaptable solution to enhance interaction with the City’s partners.” - Erika Provost, City of Henderson
  • 34. FME Flow Automations Connect different systems so actions in one system update another ● Powerful trigger mechanism via webhooks, data arrival, database triggers, etc. ● Build multi-step complex workflows ● Take advantage of true parallelism. ● Full orchestration control via split merge blocks. ● Reduce errors and save time by replacing manual tasks with automation.
  • 35. Uni-Directional Integration Architecture Linear data flow used to repair, validate, and replicate data from system A to system B ● Triggers can vary from files being received to schedules and webhooks. Dynamic data flow(s) used to integrate multiple systems by synchronizing changes between 2 (or more) systems. ● Typically driven by an action being performed in either system for near real time processing via webhooks or topics/notifications. Bi | Multi-Directional
  • 36. SFTP Dropbox System Event Azure Event Grid JMS UDP HTTP Request WebSocket FTP Logger SQS/SNS SMTP SQS/SNS Filter Merge S3 Bucket Network Triggers and Notifications
  • 37. FME Flow Automation Triggered (Webhook Received from Survey123) User submits Survey Integrate AGOL and Cityworks via REST API Published FME Workspace (.fmw)
  • 38. FME Flow Automation Triggered Record inserted, updated, or deleted Record inserted, updated, or deleted Published FME Workspace executes and synchronizes changes to between ArcGIS Online Features & Cityworks Work Orders
  • 39. FME Flow Automation Triggered Record inserted, updated, or deleted Record inserted, updated, or deleted Published FME Workspace executes and synchronizes changes to features from Cityworks, Salesforce, and ArcGIS Online
  • 40. Demo
  • 41.
  • 42. Customer Story City of Naperville Project Automating time-consuming processes for their Electric, Public Works, Finance, and Water departments. Solution Integrating FME with Cityworks to automate service requests and work orders. Results ● 3300+ service requests and work orders automated, saving hours of manual effort. ● Three staff with specialized coding skills free to focus on high-value tasks rather than manual work ● Increased collaboration and stronger relationships between departments “With FME, we were suddenly saying “yes!” to other departments and implementing requests quickly. Now, we’re viewed differently in our interactions. FME makes it easy for us to improve our relationships with other departments by helping improve their business processes.” - Danielle DuBose, Database Developer, City of Naperville
  • 44. Let’s start with the data ● Bounded data is finite and has a discrete beginning and end. It is associated with batch processing. ● Unbounded data is infinite, having no discrete beginning or end. It is associated with stream processing. ● FME handles both!
  • 45. Use Cases 1. Business Applications - Essential for driving real-time business intelligence and process automation. e.g. Customer orders, airline reservations, insurance claims, bank transactions, telco call detail records (CDR). 2. Digital Information - Captures the dynamics of online engagement and system health. e.g. clickstreams, logs, market data, email. 3. Internet of Things (IoT) - Capture the vast and growing network of connected devices, translating real-time data from the physical world into actionable intelligence. e.g RFID, telemetry SCADA, GPS sensors, smart meters, smart city infrastructure.
  • 47. Simple Streaming Pattern: Filter by Attribute Reduce data volumes by saving only what you are interested in.
  • 48. FME Flow Streams Process data continuously with low latency.
  • 50. Data Virtualization Create Open API compliant APIs on top of all FME Data Types. Real-time data (Zero-ETL) to deliver direct data access via web API. Effectively making any data cloud native.
  • 51. Data Virtualization Value Agility ● Unified view isolates applications from underlying systems. Cost Savings ● Simplify infrastructure. Save time and resources on data integration. Improved Decision Making ● Real-time access to enterprise applications used by decision makers. Enhanced Data Security and Compliance ● Data access via a secure API improves data security and governance. Increased Productivity ● Frees IT to focus on strategic projects; business applications access data directly.
  • 52. Data Virtualization Connects Anything Caching. We don’t want to run any job unless needed. Security. controlling who and type of access they have. Monitoring Who is using what how much. Rate limiting, etc. FME Flow
  • 54. FME Form Workbench - Data Virtualization
  • 56. 29+ 27K+ 128 190 20K+ years of solving data challenges FME Community members countries with FME customers organizations worldwide global partners with FME services 30+ 29K+ 128 140+ 25K+ years of solving data challenges FME Community members countries with FME customers organizations worldwide global partners with FME services 200K+ users worldwide Safe & FME
  • 58. Get our Ebook Spatial Data for the Enterprise fme.ly/gzc Guided learning experiences at your fingertips academy.safe.com FME Academy Resources Check out how-to’s & demos in the knowledge base community.safe.com /s/knowledge-base Knowledge Base Webinars Upcoming & on-demand webinars safe.com/webinars
  • 60. We’d love to help you get started. Get in touch with us at info@safe.com Experience the FME Accelerator Contact Us A world where data is not just a commodity but a catalyst for real change. fme.safe.com/accelerator Next Steps
  • 61. ClaimYour Community Badge & Dive into the new Community! ● Get community badges for watching webinars ● community.safe.com ● Today’s code: C35A62 Join the Community today!
  • 62. Q&A
  • 63. ThankYou Recap of Next Steps 1 Join the FME Community 2 Contact us 3 Experience the FME Accelerator Please fill out our webinar survey