Moving Data & Infrastructure to the Cloud
Cloud Migration
Agenda
● Who we are
● Cloud benefits and opportunities
● Considerations and planning
● Application migration
● Data migration
Meet the Presenters
Jen Luther Thomas Stewart Harper Don Murray
Company Profile
Safe Software
maximizing the value of data
25+ years
10,000+ Organizations
trusting us worldwide
delivering FME Expertise worldwide
150+ Partners
128 Countries
having deployed FME
FME®
Integration Platform
Connect. Transform. Automate. All with no code.
FME Desktop FME Server FME Cloud
Build & Run Data Workflows Automate Data Workflows
(on-premises)
Automate Data Workflows
(cloud)
Get a free trial of FME Desktop and FME Server at safe.com
Unrivaled Data Support - Key to Data and Application Migration
FME is the only
Enterprise Integration Platform
with comprehensive
spatial data support.
With the FME Platform you can…
Save time with a no-code approach
that makes migration of all kinds easier.
Save money with one tool for all types
of data and applications.
Improve data quality with built in
QA/QC tools that can enhance/fix data
issues during the migration process.
Make things possible with unrivalled
deployment and support so you can
stay in control while you migrate.
Benefits & Opportunities
of Cloud Migration
Benefits of Migrating
to the Cloud
● Flexibility and scalability
● Optimised costs
● Enhanced security and
compliance
● Backup, recovery and failover
● Simplified management and
monitoring
Opportunities of Migrating to the Cloud
Moving applications and systems to the cloud gives one of those rare
opportunities to look at all your applications.
● What applications do you want to move as is?
● What applications do you want to retire?
● Are there applications that are slow that you want to fix?
● What applications do you want to replace with Cloud based
applications?
● Do you want to enhance the data or its quality while you move it?
Poll: Where are you in your
cloud migration journey?
Considerations & Planning
Migrating to the Cloud is more
than a digital transformation,
it is a mindset transformation.
Potential Risks of
Cloud Migration
● Time and cost of cloud migration
● Cloud security and compliance
○ Data sensitivity
● Application interoperability
● Business downtime
● Organizational adoption and
training
Mitigating the Risks
● Team engagement and
training
● Legal considerations on
data storage
● Cloud vendor and region
choice
Team Engagement
● Engage team early so they
are part of the journey
● Get them the education
they need
● Time up front to get them
onside pays huge dividends
● Knowledge of existing
systems is critical.
Legal Requirements
Are there legal requirements on your
data?
● Are you storing personally
identifiable information?
● Do you need to store your data in a
certain geographic location?
● Do you need a certain compliance?
Cloud Vendor Choice
What services do you need?
Are your staff trained?
Is it compatible with your existing
architecture?
Ensure vendor meets all your legal and
security requirements.
Does the vendor have a data center in
your region?
Cloud Vendor by Adoption
Poll: What cloud vendors
are you migrating to?
Application Migration
Example Web Application
Web Layer
Application Layer
Scalable Worker Nodes
Storage
Rehost (Lift and Shift )
Lift entire application from internal servers and move to the cloud without
making any changes to the code, deployment or OS.
Useful for
● Simple or Legacy applications with few integrations and dependencies
● Applications for which you need to migrate quickly, with minimal
downtime.
FME Server Example
On Premises Azure Cloud
Benefits & Drawbacks
Benefits
● The most straightforward option.
● Often the least upfront cost
● Low risk as no application change.
● Skill often align with existing skills.
● Potential for improved performance.
Risks
● Doesn’t usually take full advantage
of the cloud (e.g. Autoscaling)
● Not guaranteed to run as well on
cloud as running on public
infrastructure.
● Poor cost optimization
Re-Platforming (Lift and Shape)
Retool an application without changing its core architecture.
Make a few cloud optimizations to ensure the app make the best use of cloud.
Use when you can identify a clear ROI for adding new features in the cloud.
Avoids the expense and downtime of a full re-architecting.
FME Server Azure Marketplace
}
Cloud Marketplaces
Deploy FME Server into your own cloud infrastructure.
FME Platform (BYOL)
FME Platform (BYOL)
Distributed FME Server Deployment
FME Server Fargate Deployment
Benefits & Drawbacks
Benefits
● Less time and money than
re-architecting. Faster ROI.
● Lower risk than re-architecting as you
address one area of the application at
a time.
● Leverage 3rd party SaaS components
e.g. Auth0.
Risks
● Scope creep can lead to it essentially
becoming a re-architecture.
● Need to understand cloud-native
tools to make good decisions.
Re-architecting
Changing significant portions of your application’s
codebase to fit the cloud environment.
Considerations
Use for applications that are not compatible with the cloud for architectural reasons, or
to innovate and add new features.
● Best-suited to long-term projects where you can take advantage of the ROI, and
where the that justifies the initial cost.
● Make conscious decisions on where to lock in with cloud-specific features (AWS
Lambda, Azure functions) and where to use cloud-agnostic technology such as
Kubernetes.
Re-architecting a Complex Web Application Source: Azure introduction to cloud-native applications
Benefits & Drawbacks
Benefits
● Long-term cost optimization.
● Replace outdated code.
● Modern architecture giving you:
○ Increased agility
○ Improved security
○ Stability
Drawbacks
● The most costly and time consuming.
● Higher risk. Updating legacy codebase.
● New skills needed(e.g. coding,
automation, DevOps)
● May need to replace some commercial
software components
● Potential for vendor lock-in
● Data migration challenges
Replace
Retire the application and replace it with a new
cloud-native application.
Considerations
A replacement needs to be identified and deployed.
Look for SaaS equivalent that can provide the functionality of the hosted application
without any development, management or infrastructure overheads.
Replacing is the right path when an application cannot be run in the cloud environment
and a rebuild/re architect isn’t viable.
Replacing FME Server with FME Cloud
Safe moving to UKG. Always look to see if you can replace and simplify.
Benefits & Drawbacks
Benefits
- Can be simple low risk option so
long as you can migrate the data.
- Gain all of the advantages of the
cloud.
- Access to new features.
Drawbacks
- May need to terminate existing
license agreements.
- May lose some functionality.
- Staff training is important and can’t
be ignored.
- Data migration challenges.
Retire
Can the application or infrastructure be turned off without any
decrease in productivity or value loss for the end consumers?
Maybe it’s time to simply retire the application.
Application Migration
Summary
● Moving applications can be difficult and
needs to be planned.
● Select strategy for each application area.
● A slow controlled fashion will need a data
migration strategy.
● Big bang approach often not possible.
● Opportunity to modernize to true cloud
based deployments. Kubernetes,
Serverless, SaaS, etc.
Data Migration
Applications run on data.
Data also needs to be migrated.
Considerations
Data Gravity
Key Point #1: For optimal performance,
applications and data need to be as
close as possible.
Using Cloud processing for on premises
data or vice versa may not be workable
for production.
Tip: Choose applications that are close
to your data.
Data Gravity
Key Point #2: For optimal user
experience the location of users and
applications should be as close as
possible.
Jen
Region or Data Center
Ensure you pick tools that can be
located close to the data.
For migration it is often useful to have a
a tool that can be run both on premises
and in the cloud.
Migrations
To avoid or reduce latency, set up your services
or data storage as close to users and applications as possible.
Google Cloud Platform’s locations and network
Data Gravity
Database Migration
Are you staying with the same database or moving to a cloud
native database?
Database Migration Example
Lift and Shift
● Leverage database
migration tools.
Re-platform (PaaS)
● Leverage database
migration tools or cloud
migration services.
Replace
● ETL tool to migrate
data.
● Data cleanup as part
of migration.
Data Migration
Storage type(s): this is dictated by the application or service that you are going to
use, Object/File/Block
Geographic Location: Where are the applications running?
Costs: Storage costs in the cloud are lower than you would pay on premises. Costs
rise based on volume and performance.
Access Frequency: How often will you need to access your data? Storage services
can be optimized for this.
Application Integration: Do the applications or services that you’ll be connecting
to your data have their own requirements?
Considerations
Data Migration
Choosing the right service is important
● Object storage
● Store and protect
any amount of
data
● Scalability,
availability,
durability
● Easy to manage
● Elastic File
System
● NFS
● Fully managed
● Scalable, elastic
● Linux
● Elastic Block
Storage
● Designed to use
with EC2
● Easy to scale
storage and
performance
● High
performance
● Fully managed
● Storage for
compute
workloads,
machine learning,
media processing
● Connects to S3
● Windows File
Server
● SMB
● Fully managed
● Fast and flexible
performance
● AWS DataSync
API/Service Migration
ETL
● Extract data in some format from old system
● Transform data to meet schema, and file requirements, optionally clean up data
along the way.
● Load resulting data into new system
APIs
● Use APIs to read the data from the old system
● Transform to meet schema/data requirements, with optional cleanup
● Use APIs to put the data into the new system
Migrating to a new cloud service? How do you get existing data
into a new platform or service?
Why use FME for
migration?
● Wide system support - text field, csv,
json, databases
● Ability to connect to APIs
● Data transformation, QA/QC and data
validation capabilities
● Automate testing and production
processes
● Log and notify stakeholders of errors
every step of the way
● Scale up and parallelize migration
with FME Server
Why use FME for
migration?
Support for major cloud provider services
Why use FME for migration?
Support for many cloud services
And many more......
Data Synchronization
Data Synchronization
● May require service downtime or ‘read-only’ during migration
● How up-to-date do you need data to be?
○ Schedule data synchronization frequency
○ Schedule out of business hours
○ Use APIs for real-time data synchronization
How can you keep your data in sync during a migration?
How can you keep your data in sync between systems?
How do you keep data in sync with a hybrid model?
Data Synchronization
● Automate synchronization
○ FME Server triggers - automatically synchronize data that appears in a
folder / S3 bucket etc
○ Use webhooks
○ Use APIs
○ FME Server can easily run workflows triggered by cloud compute services
(lambda, functions)
Use FME Server for data synchronization
Migrating to the cloud requires a
mindset shift to be successful
Use the key strategies presented
here to migrate your applications
Use the opportunity to improve
your data and applications
Tools like FME and is unrivalled
system support are key to a
successful migration.
Summary
Benefits of
the cloud
Look for FME in your cloud marketplace.
Thank you!
Connect with us for more FME

Cloud Migration: Moving Data and Infrastructure to the Cloud

  • 1.
    Moving Data &Infrastructure to the Cloud Cloud Migration
  • 2.
    Agenda ● Who weare ● Cloud benefits and opportunities ● Considerations and planning ● Application migration ● Data migration
  • 3.
    Meet the Presenters JenLuther Thomas Stewart Harper Don Murray
  • 4.
    Company Profile Safe Software maximizingthe value of data 25+ years 10,000+ Organizations trusting us worldwide delivering FME Expertise worldwide 150+ Partners 128 Countries having deployed FME
  • 5.
    FME® Integration Platform Connect. Transform.Automate. All with no code. FME Desktop FME Server FME Cloud Build & Run Data Workflows Automate Data Workflows (on-premises) Automate Data Workflows (cloud) Get a free trial of FME Desktop and FME Server at safe.com
  • 6.
    Unrivaled Data Support- Key to Data and Application Migration
  • 7.
    FME is theonly Enterprise Integration Platform with comprehensive spatial data support.
  • 8.
    With the FMEPlatform you can… Save time with a no-code approach that makes migration of all kinds easier. Save money with one tool for all types of data and applications. Improve data quality with built in QA/QC tools that can enhance/fix data issues during the migration process. Make things possible with unrivalled deployment and support so you can stay in control while you migrate.
  • 9.
  • 10.
    Benefits of Migrating tothe Cloud ● Flexibility and scalability ● Optimised costs ● Enhanced security and compliance ● Backup, recovery and failover ● Simplified management and monitoring
  • 11.
    Opportunities of Migratingto the Cloud Moving applications and systems to the cloud gives one of those rare opportunities to look at all your applications. ● What applications do you want to move as is? ● What applications do you want to retire? ● Are there applications that are slow that you want to fix? ● What applications do you want to replace with Cloud based applications? ● Do you want to enhance the data or its quality while you move it?
  • 12.
    Poll: Where areyou in your cloud migration journey?
  • 13.
  • 14.
    Migrating to theCloud is more than a digital transformation, it is a mindset transformation.
  • 15.
    Potential Risks of CloudMigration ● Time and cost of cloud migration ● Cloud security and compliance ○ Data sensitivity ● Application interoperability ● Business downtime ● Organizational adoption and training
  • 16.
    Mitigating the Risks ●Team engagement and training ● Legal considerations on data storage ● Cloud vendor and region choice
  • 17.
    Team Engagement ● Engageteam early so they are part of the journey ● Get them the education they need ● Time up front to get them onside pays huge dividends ● Knowledge of existing systems is critical.
  • 18.
    Legal Requirements Are therelegal requirements on your data? ● Are you storing personally identifiable information? ● Do you need to store your data in a certain geographic location? ● Do you need a certain compliance?
  • 19.
    Cloud Vendor Choice Whatservices do you need? Are your staff trained? Is it compatible with your existing architecture? Ensure vendor meets all your legal and security requirements. Does the vendor have a data center in your region?
  • 20.
  • 21.
    Poll: What cloudvendors are you migrating to?
  • 22.
  • 23.
    Example Web Application WebLayer Application Layer Scalable Worker Nodes Storage
  • 25.
    Rehost (Lift andShift ) Lift entire application from internal servers and move to the cloud without making any changes to the code, deployment or OS. Useful for ● Simple or Legacy applications with few integrations and dependencies ● Applications for which you need to migrate quickly, with minimal downtime.
  • 26.
    FME Server Example OnPremises Azure Cloud
  • 27.
    Benefits & Drawbacks Benefits ●The most straightforward option. ● Often the least upfront cost ● Low risk as no application change. ● Skill often align with existing skills. ● Potential for improved performance. Risks ● Doesn’t usually take full advantage of the cloud (e.g. Autoscaling) ● Not guaranteed to run as well on cloud as running on public infrastructure. ● Poor cost optimization
  • 29.
    Re-Platforming (Lift andShape) Retool an application without changing its core architecture. Make a few cloud optimizations to ensure the app make the best use of cloud. Use when you can identify a clear ROI for adding new features in the cloud. Avoids the expense and downtime of a full re-architecting.
  • 30.
    FME Server AzureMarketplace }
  • 31.
    Cloud Marketplaces Deploy FMEServer into your own cloud infrastructure. FME Platform (BYOL) FME Platform (BYOL) Distributed FME Server Deployment
  • 32.
  • 33.
    Benefits & Drawbacks Benefits ●Less time and money than re-architecting. Faster ROI. ● Lower risk than re-architecting as you address one area of the application at a time. ● Leverage 3rd party SaaS components e.g. Auth0. Risks ● Scope creep can lead to it essentially becoming a re-architecture. ● Need to understand cloud-native tools to make good decisions.
  • 35.
    Re-architecting Changing significant portionsof your application’s codebase to fit the cloud environment.
  • 36.
    Considerations Use for applicationsthat are not compatible with the cloud for architectural reasons, or to innovate and add new features. ● Best-suited to long-term projects where you can take advantage of the ROI, and where the that justifies the initial cost. ● Make conscious decisions on where to lock in with cloud-specific features (AWS Lambda, Azure functions) and where to use cloud-agnostic technology such as Kubernetes.
  • 37.
    Re-architecting a ComplexWeb Application Source: Azure introduction to cloud-native applications
  • 38.
    Benefits & Drawbacks Benefits ●Long-term cost optimization. ● Replace outdated code. ● Modern architecture giving you: ○ Increased agility ○ Improved security ○ Stability Drawbacks ● The most costly and time consuming. ● Higher risk. Updating legacy codebase. ● New skills needed(e.g. coding, automation, DevOps) ● May need to replace some commercial software components ● Potential for vendor lock-in ● Data migration challenges
  • 40.
    Replace Retire the applicationand replace it with a new cloud-native application.
  • 41.
    Considerations A replacement needsto be identified and deployed. Look for SaaS equivalent that can provide the functionality of the hosted application without any development, management or infrastructure overheads. Replacing is the right path when an application cannot be run in the cloud environment and a rebuild/re architect isn’t viable.
  • 42.
    Replacing FME Serverwith FME Cloud
  • 43.
    Safe moving toUKG. Always look to see if you can replace and simplify.
  • 44.
    Benefits & Drawbacks Benefits -Can be simple low risk option so long as you can migrate the data. - Gain all of the advantages of the cloud. - Access to new features. Drawbacks - May need to terminate existing license agreements. - May lose some functionality. - Staff training is important and can’t be ignored. - Data migration challenges.
  • 46.
    Retire Can the applicationor infrastructure be turned off without any decrease in productivity or value loss for the end consumers? Maybe it’s time to simply retire the application.
  • 47.
    Application Migration Summary ● Movingapplications can be difficult and needs to be planned. ● Select strategy for each application area. ● A slow controlled fashion will need a data migration strategy. ● Big bang approach often not possible. ● Opportunity to modernize to true cloud based deployments. Kubernetes, Serverless, SaaS, etc.
  • 48.
    Data Migration Applications runon data. Data also needs to be migrated.
  • 49.
  • 50.
    Data Gravity Key Point#1: For optimal performance, applications and data need to be as close as possible. Using Cloud processing for on premises data or vice versa may not be workable for production. Tip: Choose applications that are close to your data.
  • 51.
    Data Gravity Key Point#2: For optimal user experience the location of users and applications should be as close as possible. Jen
  • 52.
    Region or DataCenter Ensure you pick tools that can be located close to the data. For migration it is often useful to have a a tool that can be run both on premises and in the cloud.
  • 53.
  • 54.
    To avoid orreduce latency, set up your services or data storage as close to users and applications as possible. Google Cloud Platform’s locations and network Data Gravity
  • 55.
    Database Migration Are youstaying with the same database or moving to a cloud native database?
  • 56.
    Database Migration Example Liftand Shift ● Leverage database migration tools. Re-platform (PaaS) ● Leverage database migration tools or cloud migration services. Replace ● ETL tool to migrate data. ● Data cleanup as part of migration.
  • 57.
    Data Migration Storage type(s):this is dictated by the application or service that you are going to use, Object/File/Block Geographic Location: Where are the applications running? Costs: Storage costs in the cloud are lower than you would pay on premises. Costs rise based on volume and performance. Access Frequency: How often will you need to access your data? Storage services can be optimized for this. Application Integration: Do the applications or services that you’ll be connecting to your data have their own requirements? Considerations
  • 58.
    Data Migration Choosing theright service is important ● Object storage ● Store and protect any amount of data ● Scalability, availability, durability ● Easy to manage ● Elastic File System ● NFS ● Fully managed ● Scalable, elastic ● Linux ● Elastic Block Storage ● Designed to use with EC2 ● Easy to scale storage and performance ● High performance ● Fully managed ● Storage for compute workloads, machine learning, media processing ● Connects to S3 ● Windows File Server ● SMB ● Fully managed ● Fast and flexible performance ● AWS DataSync
  • 59.
    API/Service Migration ETL ● Extractdata in some format from old system ● Transform data to meet schema, and file requirements, optionally clean up data along the way. ● Load resulting data into new system APIs ● Use APIs to read the data from the old system ● Transform to meet schema/data requirements, with optional cleanup ● Use APIs to put the data into the new system Migrating to a new cloud service? How do you get existing data into a new platform or service?
  • 60.
    Why use FMEfor migration? ● Wide system support - text field, csv, json, databases ● Ability to connect to APIs ● Data transformation, QA/QC and data validation capabilities ● Automate testing and production processes ● Log and notify stakeholders of errors every step of the way ● Scale up and parallelize migration with FME Server
  • 61.
    Why use FMEfor migration? Support for major cloud provider services
  • 62.
    Why use FMEfor migration? Support for many cloud services And many more......
  • 63.
  • 64.
    Data Synchronization ● Mayrequire service downtime or ‘read-only’ during migration ● How up-to-date do you need data to be? ○ Schedule data synchronization frequency ○ Schedule out of business hours ○ Use APIs for real-time data synchronization How can you keep your data in sync during a migration? How can you keep your data in sync between systems? How do you keep data in sync with a hybrid model?
  • 65.
    Data Synchronization ● Automatesynchronization ○ FME Server triggers - automatically synchronize data that appears in a folder / S3 bucket etc ○ Use webhooks ○ Use APIs ○ FME Server can easily run workflows triggered by cloud compute services (lambda, functions) Use FME Server for data synchronization
  • 66.
    Migrating to thecloud requires a mindset shift to be successful Use the key strategies presented here to migrate your applications Use the opportunity to improve your data and applications Tools like FME and is unrivalled system support are key to a successful migration. Summary Benefits of the cloud
  • 67.
    Look for FMEin your cloud marketplace. Thank you! Connect with us for more FME