SlideShare a Scribd company logo
Modernizing Hazard Data Feeds
at the Pacific Disaster Center
20
22
FME
User
Conference
● Introduction
● About Kontur and the Pacific Disaster Center (PDC)
● Review of current data processing methods at PDC
● Data modernization with FME
● Examples of FME in use at PDC
● Additional infrastructure changes with FME
● The path forward for PDC and FME
Agenda
20
22
FME
User
Conference
Jorma Rodieck
o Worked in the IT field for the last 20+ years
o Took my first GIS class at UC Santa Cruz in 1996
o Worked with Kontur last seven years primarily with the PDC
o Strong technical background with a focus on GIS related
projects the last 12 years
o Bachelor’s degree in Biology from UC Santa Cruz
o Post-Baccalaureate Certificate in Geographic Information
Systems from Penn State University
o AWS Certified Developer – Associate
o From the Pacific NW and currently living in Maui, Hawaii
Project Manager / Developer at Kontur
20
22
FME
User
Conference
20
22
FME
User
Conference
Kontur is a company specialized in disaster management and geoinformation.
Our solutions include datasets and program products that are used in
emergencies.
Kontur helps customers:
● track events in real-time
● estimate risks and impact
● get notified about changes in the course of events
About Kontur
20
22
FME
User
Conference
20
22
FME
User
Conference
© Copyright PDC 2006-2022
Who is PDC?
Staff and industry experts
of disasters
8 Global
Locations
Thousands
More than 60
Managed by
University of
Hawaii
since 2006
200+
Partner projects and
engagements
25
YEARS
OF SERVICE
+
Since 1996
Over 70
Partner Countries
PDC
Headquarters
www.pdc.org
20
22
FME
User
Conference
© Copyright PDC 2006-2022 www.pdc.org
Who uses our systems?
Our partners at-a-glance
National Governments
United States Agencies International & Regional Orgs
20
22
FME
User
Conference
© Copyright PDC 2006-2022
Our Global Mission
www.pdc.org
To provide evidence-based research, applied science, information, and cutting-edge technology
solutions for more effective disaster risk reduction (DRR) policies, practices, and humanitarian
assistance and disaster relief (HA/DR) operations in the Asia Pacific region and beyond.
20
22
FME
User
Conference
© Copyright PDC 2006-2022 www.pdc.org
Disasters happen fast. Make better decisions faster.
Real-time and
historical hazards
Anticipated
impacts
Collaboration and
sharing
Live
cameras
Smart AlertTM
early warning
Asset
protection
Unrivaled
Situational
Awareness
20
22
FME
User
Conference
© Copyright PDC 2006-2022
Disasters happen fast. Make better decisions faster.
6,500+
DATA LAYERS
20
22
FME
User
Conference
© Copyright PDC 2006-2022
Disasters happen fast. Make better decisions faster.
Disaster Alert™
disasteralert.pdc.org
DisasterAWARE Pro®
disasteraware.pdc.org
Customizable Solutions
Unlock the power of your own data.
Disaster Alert is a free, public
application that provides individuals,
families, and their loved ones with the
information they need to stay safe
anywhere in the world. Built on PDC s
DisasterAWARE® platform, Disaster Alert
offers real-time updates on hazards as
they are unfolding around the globe.
DisasterAWARE is the most powerful and
reliable early warning and multi-
hazard monitoring platform. It is used
by thousands of the most demanding
government customers and humanitarian
assistance organizations around the globe.
It supports disaster risk reduction and best
practices throughout all phases of disaster
management—providing early warning,
multi-hazard monitoring, modeled impact
assessments, and the largest collection of
scientifically verified geospatial data.
We provide out-of-the box and customized
disaster management solutions for early
warning and decision support, built on our
industry-leading disaster risk intelligence
platform, DisasterAWARE.
Learn more about our tailored DisasterAWARE
solutions.
Informing Decision-Making Worldwide
20
22
FME
User
Conference
How is PDC using FME?
PDC’s Implementation:
o PDC aggregates various hazard and data feeds
from a large variety of sources
o The data aggregation work is primarily done by a
custom Java application built by PDC 10+ years ago
o Issues arise with this process due to:
o Complexity of the Java application
o Monolithic architecture
o Scalability issues
o Limited knowledge of how the application is put
together and operates
o High level of programming experience needed
to make changes to the application
20
22
FME
User
Conference
How is PDC using FME?
Modernization of PDC’s Hazard Data Aggregation:
o Installation of FME Desktop and FME Server
o Test and Production Environments running FME Server
o Move automation process from legacy Java application
to FME Server
o Review of all logic and algorithms used in data
collection and aggregation for each hazard type
o Rewrite data aggregation and processing logic using
FME
o Review of FME Desktop results to confirm they
conform to expected known values
o Publish to FME Server Test
o Integrate into PDC products and services
20
22
FME
User
Conference
2) Publish to FME Server
FME Data Process at PDC
1) Workbench creation using FME Desktop
3) FME Server writes data
to PostgreSQL DB
4) Map document creation
using data in SQL DB
5) Publish map document to create map
service in ArcGIS Server
From FME to PDC’s DisasterAWARE Application
6) Create new layer in
DisasterAWARE
20
22
FME
User
Conference
Example: Wildfire Exposure
Providing real-time hazard information
1. Connect to NIFC’s API for real-time fire perimeter information
2. Connect the fire perimeter to real-time fire hazard information
3. Build products for information decision-makers
20
22
FME
User
Conference
Example: Public Weather Alerts for Canada
1. PDC Monitors an RSS feed and
processes the data every time
a new alert is issued
2. Each hazard received contains
polygon geometry and
attributes such as title, author,
description, event, affected
areas, severity.
3. Processing is done within FME
and a new hazard is created
within DisasterAWARE if the
specific criteria is met for that
hazard type.
4. Additional processing done in
FME to create exposure
impact data (see next slide)
20
22
FME
User
Conference
Example:
FME Integration into PDC’s Hazard Brief
1. Hazard Brief is a DisasterAWARE
component that contains the latest
hazard impact data for a specific
hazard
2. Hazard Brief updates the hazard
data shown as new data comes
available
3. FME is utilized in Hazard Brief
processing for calculating the
exposure impact from the hazard
polygon and providing the
exposure data to the Hazard Brief
application
20
22
FME
User
Conference
Additional FME Infrastructure Changes
Monitoring and Alerting
o Leveraging the FME Server API and Google Sheets:
o Daily FME job to compare Test and Prod FME Servers to make sure that
workbenches, automations, schedules, etc. are in sync
o Daily FME job to confirm that workspaces on the server are the same as the
files checked into the PDC Git repo
o Run checks on critical jobs to report if they have failed x times in a row
o Example: It is ok for a job that runs every 10min to fail once, but not ok for a job that
runs once a month to fail once
o Run regular checks on queues to ensure jobs are processed efficiently
o Integration into PDC’s system monitoring tool (Site24x7) to provide immediate
alerting for critical issues
o Slack and email notifications for job failures and configuration changes
20
22
FME
User
Conference
How has FME improved life at PDC?
o FME helps remove much of the complexity when making
changes to existing code
o Less dependency on highly technical Java developers
o No longer need to redeploy the whole application when
making changes
o Improved reliability: A bug in an FME workspace will not
impact other workspaces
o Separation of jobs into individual workbenches makes it
easier to understand each component
o Knowledge of how each workbench operates can be
easily shared
o Lowers the barrier to entry for skills required to develop
o Easier to debug and rerun jobs when something is not
working
20
22
FME
User
Conference
The Path Forward for PDC and FME
PDC plans to migrate all existing data processes
from their existing monolithic Java application to
FME. The logic and data science used for
aggregating hazard data for each hazard type will
be reviewed, and then after the logic and
algorithms are confirmed all data processing for
the hazard type will be moved to FME. Once
completed for all hazard types, the existing Java
application will be removed from production.
20
22
FME
User
Conference
Questions? Comments?
20
22
FME
User
Conference
Resources
Pacific Disaster Center: https://pdc.org
Kontur, Inc: https://kontur.io
DisasterAlert (PDC’s public DisasterAWARE application): https://disasteralert.pdc.org
DisasterAlert Mobile:
Thank You!
jorma@kontur.io
20
22
FME
User
Conference
Please Share Your Session Feedback
Log in & navigate to the
agenda. Select the
session you attended
Scroll down to “Session
Feedback”
Download EventMobi app
Event code: fmeuc
Click “Launch App”
Fill out the survey
and submit

More Related Content

Similar to Modernizing Hazard Data Feeds at the PDC

Resume_Hegel_updated_Network (1)
Resume_Hegel_updated_Network (1)Resume_Hegel_updated_Network (1)
Resume_Hegel_updated_Network (1)
Hegel Durocher
 
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
Deltares
 
Resume_Padmaja
Resume_PadmajaResume_Padmaja
Resume_Padmaja
Padmaja Samantara
 
Automated Application Integration with FME & Cityworks Webinar
Automated Application Integration with FME & Cityworks WebinarAutomated Application Integration with FME & Cityworks Webinar
Automated Application Integration with FME & Cityworks Webinar
Safe Software
 
How Utilities can Improve Operations using Data Integration Workflows
How Utilities can Improve Operations using Data Integration WorkflowsHow Utilities can Improve Operations using Data Integration Workflows
How Utilities can Improve Operations using Data Integration Workflows
Safe Software
 
Mud Reporting: Streamlined Process and Data Management
Mud Reporting: Streamlined Process and Data ManagementMud Reporting: Streamlined Process and Data Management
Mud Reporting: Streamlined Process and Data Management
pvisoftware
 
Michael Haftel Resume-2016
Michael Haftel Resume-2016Michael Haftel Resume-2016
Michael Haftel Resume-2016
Michael Haftel
 
Amer-CV-2014_10
Amer-CV-2014_10Amer-CV-2014_10
Amer-CV-2014_10
amer salameh
 
8 Ways Utility Networks Can Meet Data Demands
8 Ways Utility Networks Can Meet Data Demands8 Ways Utility Networks Can Meet Data Demands
8 Ways Utility Networks Can Meet Data Demands
Safe Software
 
Amit Chaubey
Amit ChaubeyAmit Chaubey
Amit Chaubey
Amit Chaubey
 
Application Specialist
Application SpecialistApplication Specialist
Application Specialist
Brian Machado
 
cv15
cv15cv15
Resume2_Roy Anderson_online version
Resume2_Roy Anderson_online versionResume2_Roy Anderson_online version
Resume2_Roy Anderson_online version
Roy Anderson
 
Torben Verdich Resume TS-SCI
Torben Verdich Resume TS-SCITorben Verdich Resume TS-SCI
Torben Verdich Resume TS-SCI
Torben Verdich, MCITP
 
Current_CV
Current_CVCurrent_CV
Mindtree's performance engineering services.
Mindtree's performance engineering services.Mindtree's performance engineering services.
Mindtree's performance engineering services.
Mindtree Ltd.
 
Avoiding disaster recovery disasters
Avoiding disaster recovery disastersAvoiding disaster recovery disasters
Avoiding disaster recovery disasters
Alexandra Matthiesen
 
Avoiding disaster recovery disasters
Avoiding disaster recovery disastersAvoiding disaster recovery disasters
Avoiding disaster recovery disasters
Veritas Technologies LLC
 
Production Deployment of the New Integrator Project C.docx
Production Deployment of the New Integrator  Project C.docxProduction Deployment of the New Integrator  Project C.docx
Production Deployment of the New Integrator Project C.docx
briancrawford30935
 
Chuck_Roden_Resume
Chuck_Roden_ResumeChuck_Roden_Resume
Chuck_Roden_Resume
Chuck Roden
 

Similar to Modernizing Hazard Data Feeds at the PDC (20)

Resume_Hegel_updated_Network (1)
Resume_Hegel_updated_Network (1)Resume_Hegel_updated_Network (1)
Resume_Hegel_updated_Network (1)
 
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
 
Resume_Padmaja
Resume_PadmajaResume_Padmaja
Resume_Padmaja
 
Automated Application Integration with FME & Cityworks Webinar
Automated Application Integration with FME & Cityworks WebinarAutomated Application Integration with FME & Cityworks Webinar
Automated Application Integration with FME & Cityworks Webinar
 
How Utilities can Improve Operations using Data Integration Workflows
How Utilities can Improve Operations using Data Integration WorkflowsHow Utilities can Improve Operations using Data Integration Workflows
How Utilities can Improve Operations using Data Integration Workflows
 
Mud Reporting: Streamlined Process and Data Management
Mud Reporting: Streamlined Process and Data ManagementMud Reporting: Streamlined Process and Data Management
Mud Reporting: Streamlined Process and Data Management
 
Michael Haftel Resume-2016
Michael Haftel Resume-2016Michael Haftel Resume-2016
Michael Haftel Resume-2016
 
Amer-CV-2014_10
Amer-CV-2014_10Amer-CV-2014_10
Amer-CV-2014_10
 
8 Ways Utility Networks Can Meet Data Demands
8 Ways Utility Networks Can Meet Data Demands8 Ways Utility Networks Can Meet Data Demands
8 Ways Utility Networks Can Meet Data Demands
 
Amit Chaubey
Amit ChaubeyAmit Chaubey
Amit Chaubey
 
Application Specialist
Application SpecialistApplication Specialist
Application Specialist
 
cv15
cv15cv15
cv15
 
Resume2_Roy Anderson_online version
Resume2_Roy Anderson_online versionResume2_Roy Anderson_online version
Resume2_Roy Anderson_online version
 
Torben Verdich Resume TS-SCI
Torben Verdich Resume TS-SCITorben Verdich Resume TS-SCI
Torben Verdich Resume TS-SCI
 
Current_CV
Current_CVCurrent_CV
Current_CV
 
Mindtree's performance engineering services.
Mindtree's performance engineering services.Mindtree's performance engineering services.
Mindtree's performance engineering services.
 
Avoiding disaster recovery disasters
Avoiding disaster recovery disastersAvoiding disaster recovery disasters
Avoiding disaster recovery disasters
 
Avoiding disaster recovery disasters
Avoiding disaster recovery disastersAvoiding disaster recovery disasters
Avoiding disaster recovery disasters
 
Production Deployment of the New Integrator Project C.docx
Production Deployment of the New Integrator  Project C.docxProduction Deployment of the New Integrator  Project C.docx
Production Deployment of the New Integrator Project C.docx
 
Chuck_Roden_Resume
Chuck_Roden_ResumeChuck_Roden_Resume
Chuck_Roden_Resume
 

More from Safe Software

Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Safe Software
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
Safe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
Safe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Safe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Safe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Safe Software
 

More from Safe Software (20)

Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 

Recently uploaded

From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
Sease
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Vadym Kazulkin
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
christinelarrosa
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 

Recently uploaded (20)

From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 

Modernizing Hazard Data Feeds at the PDC

  • 1. Modernizing Hazard Data Feeds at the Pacific Disaster Center
  • 2. 20 22 FME User Conference ● Introduction ● About Kontur and the Pacific Disaster Center (PDC) ● Review of current data processing methods at PDC ● Data modernization with FME ● Examples of FME in use at PDC ● Additional infrastructure changes with FME ● The path forward for PDC and FME Agenda
  • 3. 20 22 FME User Conference Jorma Rodieck o Worked in the IT field for the last 20+ years o Took my first GIS class at UC Santa Cruz in 1996 o Worked with Kontur last seven years primarily with the PDC o Strong technical background with a focus on GIS related projects the last 12 years o Bachelor’s degree in Biology from UC Santa Cruz o Post-Baccalaureate Certificate in Geographic Information Systems from Penn State University o AWS Certified Developer – Associate o From the Pacific NW and currently living in Maui, Hawaii Project Manager / Developer at Kontur
  • 5. 20 22 FME User Conference Kontur is a company specialized in disaster management and geoinformation. Our solutions include datasets and program products that are used in emergencies. Kontur helps customers: ● track events in real-time ● estimate risks and impact ● get notified about changes in the course of events About Kontur
  • 7. 20 22 FME User Conference © Copyright PDC 2006-2022 Who is PDC? Staff and industry experts of disasters 8 Global Locations Thousands More than 60 Managed by University of Hawaii since 2006 200+ Partner projects and engagements 25 YEARS OF SERVICE + Since 1996 Over 70 Partner Countries PDC Headquarters www.pdc.org
  • 8. 20 22 FME User Conference © Copyright PDC 2006-2022 www.pdc.org Who uses our systems? Our partners at-a-glance National Governments United States Agencies International & Regional Orgs
  • 9. 20 22 FME User Conference © Copyright PDC 2006-2022 Our Global Mission www.pdc.org To provide evidence-based research, applied science, information, and cutting-edge technology solutions for more effective disaster risk reduction (DRR) policies, practices, and humanitarian assistance and disaster relief (HA/DR) operations in the Asia Pacific region and beyond.
  • 10. 20 22 FME User Conference © Copyright PDC 2006-2022 www.pdc.org Disasters happen fast. Make better decisions faster. Real-time and historical hazards Anticipated impacts Collaboration and sharing Live cameras Smart AlertTM early warning Asset protection Unrivaled Situational Awareness
  • 11. 20 22 FME User Conference © Copyright PDC 2006-2022 Disasters happen fast. Make better decisions faster. 6,500+ DATA LAYERS
  • 12. 20 22 FME User Conference © Copyright PDC 2006-2022 Disasters happen fast. Make better decisions faster. Disaster Alert™ disasteralert.pdc.org DisasterAWARE Pro® disasteraware.pdc.org Customizable Solutions Unlock the power of your own data. Disaster Alert is a free, public application that provides individuals, families, and their loved ones with the information they need to stay safe anywhere in the world. Built on PDC s DisasterAWARE® platform, Disaster Alert offers real-time updates on hazards as they are unfolding around the globe. DisasterAWARE is the most powerful and reliable early warning and multi- hazard monitoring platform. It is used by thousands of the most demanding government customers and humanitarian assistance organizations around the globe. It supports disaster risk reduction and best practices throughout all phases of disaster management—providing early warning, multi-hazard monitoring, modeled impact assessments, and the largest collection of scientifically verified geospatial data. We provide out-of-the box and customized disaster management solutions for early warning and decision support, built on our industry-leading disaster risk intelligence platform, DisasterAWARE. Learn more about our tailored DisasterAWARE solutions. Informing Decision-Making Worldwide
  • 13. 20 22 FME User Conference How is PDC using FME? PDC’s Implementation: o PDC aggregates various hazard and data feeds from a large variety of sources o The data aggregation work is primarily done by a custom Java application built by PDC 10+ years ago o Issues arise with this process due to: o Complexity of the Java application o Monolithic architecture o Scalability issues o Limited knowledge of how the application is put together and operates o High level of programming experience needed to make changes to the application
  • 14. 20 22 FME User Conference How is PDC using FME? Modernization of PDC’s Hazard Data Aggregation: o Installation of FME Desktop and FME Server o Test and Production Environments running FME Server o Move automation process from legacy Java application to FME Server o Review of all logic and algorithms used in data collection and aggregation for each hazard type o Rewrite data aggregation and processing logic using FME o Review of FME Desktop results to confirm they conform to expected known values o Publish to FME Server Test o Integrate into PDC products and services
  • 15. 20 22 FME User Conference 2) Publish to FME Server FME Data Process at PDC 1) Workbench creation using FME Desktop 3) FME Server writes data to PostgreSQL DB 4) Map document creation using data in SQL DB 5) Publish map document to create map service in ArcGIS Server From FME to PDC’s DisasterAWARE Application 6) Create new layer in DisasterAWARE
  • 16. 20 22 FME User Conference Example: Wildfire Exposure Providing real-time hazard information 1. Connect to NIFC’s API for real-time fire perimeter information 2. Connect the fire perimeter to real-time fire hazard information 3. Build products for information decision-makers
  • 17. 20 22 FME User Conference Example: Public Weather Alerts for Canada 1. PDC Monitors an RSS feed and processes the data every time a new alert is issued 2. Each hazard received contains polygon geometry and attributes such as title, author, description, event, affected areas, severity. 3. Processing is done within FME and a new hazard is created within DisasterAWARE if the specific criteria is met for that hazard type. 4. Additional processing done in FME to create exposure impact data (see next slide)
  • 18. 20 22 FME User Conference Example: FME Integration into PDC’s Hazard Brief 1. Hazard Brief is a DisasterAWARE component that contains the latest hazard impact data for a specific hazard 2. Hazard Brief updates the hazard data shown as new data comes available 3. FME is utilized in Hazard Brief processing for calculating the exposure impact from the hazard polygon and providing the exposure data to the Hazard Brief application
  • 19. 20 22 FME User Conference Additional FME Infrastructure Changes Monitoring and Alerting o Leveraging the FME Server API and Google Sheets: o Daily FME job to compare Test and Prod FME Servers to make sure that workbenches, automations, schedules, etc. are in sync o Daily FME job to confirm that workspaces on the server are the same as the files checked into the PDC Git repo o Run checks on critical jobs to report if they have failed x times in a row o Example: It is ok for a job that runs every 10min to fail once, but not ok for a job that runs once a month to fail once o Run regular checks on queues to ensure jobs are processed efficiently o Integration into PDC’s system monitoring tool (Site24x7) to provide immediate alerting for critical issues o Slack and email notifications for job failures and configuration changes
  • 20. 20 22 FME User Conference How has FME improved life at PDC? o FME helps remove much of the complexity when making changes to existing code o Less dependency on highly technical Java developers o No longer need to redeploy the whole application when making changes o Improved reliability: A bug in an FME workspace will not impact other workspaces o Separation of jobs into individual workbenches makes it easier to understand each component o Knowledge of how each workbench operates can be easily shared o Lowers the barrier to entry for skills required to develop o Easier to debug and rerun jobs when something is not working
  • 21. 20 22 FME User Conference The Path Forward for PDC and FME PDC plans to migrate all existing data processes from their existing monolithic Java application to FME. The logic and data science used for aggregating hazard data for each hazard type will be reviewed, and then after the logic and algorithms are confirmed all data processing for the hazard type will be moved to FME. Once completed for all hazard types, the existing Java application will be removed from production.
  • 23. 20 22 FME User Conference Resources Pacific Disaster Center: https://pdc.org Kontur, Inc: https://kontur.io DisasterAlert (PDC’s public DisasterAWARE application): https://disasteralert.pdc.org DisasterAlert Mobile:
  • 25. 20 22 FME User Conference Please Share Your Session Feedback Log in & navigate to the agenda. Select the session you attended Scroll down to “Session Feedback” Download EventMobi app Event code: fmeuc Click “Launch App” Fill out the survey and submit