Heterogenous Migration with DMS & SCT - Michael Russo

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Heterogenous Migration with DMS and SCT
AWS Database Migration Service

How can I get to the cloud?
How will my on-premises data migrate to the cloud?
How can I make it transparent to my users?
Afterwards, how will on-premises and cloud data interact?
How can I integrate my data assets within AWS?
Can I get help moving off of commercial databases?
How can I move data to my data lake?

What are DMS and SCT?
AWS Database Migration Service (DMS) easily and securely
migrates and/or replicate your databases and data
warehouses to AWS
AWS Schema Conversion Tool (SCT) converts your commercial
database and data warehouse schemas to open-source engines or
AWS-native services, such as Amazon Aurora and Redshift

When to use DMS and SCT?
Modernize Migrate Replicate

When to use SCT?
Modernize
Modernize your database tier
Modernize and Migrate your Data
Warehouse to Amazon Redshift
Amazon Aurora
Amazon Redshift

SCT helps with converting tables, views, and code
Sequences
User-defined types
Synonyms
Packages
Stored procedures
Functions
Triggers
Schemas
Tables
Indexes
Views
Sort and distribution keys

SCT Migration Assessment Report
• Assessment of migration
compatibility of source databases
with open-source database
engines – RDS MySQL, RDS
PostgreSQL and Aurora
• Recommends best target engine
• Provides details level of efforts to
complete migration

Database migration assessment
Connect AWS SCT to
source and target
databases
Run assessment report
Read executive
summary
Follow detailed
instructions

Components of the AWS SCT console
1. Source schema
2. Action items
3. Target schema
4. Schema element details
5. Edit window

SCT Data Extractors
Extract Data from your data warehouse and migrate to Amazon Redshift
• Extracts data through local migration agents
• Data is optimized for Redshift and saved in local files
• Files are loaded to an Amazon S3 bucket (through network or Amazon Snowball)
and then to Amazon Redshift
Amazon
Redshift
AWS SCT S3 Bucket

When to use DMS*?
Migrate
• Migrate business-critical applications
• Migrate from Classic to VPC
• Migrate data warehouse to Redshift
• Upgrade to a minor version
• Consolidate shards into Aurora
• Archive old data
• Migrate from NoSQL to SQL, SQL to
NoSQL or NoSQL to NoSQL
Targets:
Amazon
Dynamo DB
Amazon Redshift
Amazon S3
Amazon Aurora
*DMS is a HIPAA certified service
Amazon S3
Sources:

Migration Validation
Pre-migration Assessment
• Checks migration task settings
prior to launch
• Alerts for potential issues, such as
unsupported data types
Post-migration Assessment
• Validates that all data selected for
migration migrated properly

When to use DMS?
Replicate • Create cross-regions Read Replicas
• Run your analytics in the cloud
• Keep your dev/test and production
environment sync

Why use DMS and SCT?
Secure
Cost Effective
Remove Barriers
to Entry
Allow DB
Freedom
Keep a Leg in
the Cloud
Easy to Use, but
Sophisticated…
Near-Zero
Downtime

Oracle to Aurora Migration Playbook
• Topic-by-topic overview of Oracle to Aurora
PostgreSQL migrations and “hand-on” best
practices
• How to migrate from proprietary features and the
different database objects
• Migration best practices
SCT DMS Playbook
Schema Data Best Practices

What are customers saying about us?

Our Customers

Who is saying What about DMS and SCT?
"We migrated hundreds of our clients from our in-house data-center to Amazon
RDS Oracle 12c using the AWS Data Migration Service (DMS). Due to this
service, we could live-replicate the databases between our data-center and RDS
before the migration. That kept the migration down-time to the very
minimum. We are very happy with DMS and are planning to use it for Oracle
to MySQL migration next”.
”The SCT Assessment Report was the key enabler to allow us to understand the
scope of effort required to complete an Oracle to PostgreSQL migration.
What was originally thought to be a largely manual task that no one was
particularly excited about having to do became a very straight-forward quick
and easy process."
“We are in the process of migrating some databases to Amazon Aurora. The
ease by which we can do this using the AWS Database Migration Service has
simplified this process for us and enabled us to accelerate our migration
efforts. The ability to closely monitor the process, the detailed logging feature,
and the support we received from AWS have given us a great deal of confidence
in a successful migration.”

Heterogeneous Migration
Oracle private DC to RDS PostgreSQL migration
Used the AWS Schema Conversion Tool to
convert their database schema
Used on-going replication (CDC) to keep
databases in sync until they reached the cutover
window
Benefits:
• Improved reliability of the cloud environment
• Savings on Oracle licensing costs
• SCT Assessment Report let them understand
the scope of the migration

Multi-Engine Migration
Centralized Help Desk and ITSM Solutions
Moved legacy SQL Server and MySQL customer
databases to MySQL on AWS
Benefits:
• DMS could handle both engines
• Scripting helped automate the process
• Version upgrades could happen concurrently
• 6x more migrations in a set period of time

Replication to Redshift
Applied Computer Vision Company
Used on-going replication (CDC) to keep Amazon
Redshift dimension data in sync with various
source databases
Benefits:
• Secure service which required no special
approvals
• APIs allowed full automation
• Real-time replication

Oracle RAC to PostgreSQL Replication
“Our goal is to improve lives one smile at a time”…
Replicate Oracle RAC to RDS for PostgreSQL
Replicate PostgreSQL to Amazon Redshift
• Higher performance without the Oracle cost
• Ability to split the data and keep master tables
in sync
• Use SCT to begin the modernization journey
“DMS is a great product. From looking at the product to getting it in production takes no time
at all. Its simple to use and reliable. The flexibility of creating new tasks is not very time
consuming and just works.”

Partners

AWS database migration partners

How does it work?

Database migration process
Step 1: Convert or Copy your Schema
Source DB or DW
AWS SCT
Native Tool
Destination DB or DW
Step 2: Move your data
Source DB or DW
AWS SCT
Destination DB or DW
AWS DMS

Customer
premises
Application users
AWS
Internet
VPN
Start a replication instance
Connect to source and target
databases
Select tables, schemas, or
databases
 Let AWS DMS create tables,
load data, and keep them in
sync
 Switch applications over to
the target at your convenience
Keep your apps running during the migration
AWS
DMS

Homogenous or heterogeneous
SQL Server MySQL
Oracle Oracle
Oracle S3

Sources for AWS Database Migration Service
On-premises and Amazon EC2 instance databases:
• Oracle Database 10g, 11g, 12c
• Microsoft SQL Server 2005, 2008, 2008R2, 2012, 2014, and 2016
• MySQL 5.5, 5.6, 5.7
• MariaDB (as MySQL-compatible data source)
• PostgreSQL 9.4 and later
• SAP ASE 12.5.3, 15, 15.5, 15.7, 16 and later
• MongoDB 2.6.x, 3.x and later
• Db2 LUW 9.7, 10.1, 10.5
Cloud instance databases:
• Oracle Database 11g and 12c
• Microsoft SQL Server 2008R2, 2012, 2014, and 2016
• MySQL versions 5.5, 5.6, and 5.7
• Amazon Aurora (MySQL-compatible data source)
• Amazon S3
• Azure SQL Database

Targets for AWS Database Migration Service
On-premises and Amazon EC2 instance databases:
• Oracle Database 10g, 11g, 12c
• Microsoft SQL Server 2005, 2008, 2008R2, 2012, 2014, and 2016
• MySQL 5.5, 5.6, 5.7
• SAP ASE 15, 15.5, 15.7, 16 and later
Cloud instance databases:
• Oracle Database 11g and 12c
• Microsoft SQL Server 2008R2, 2012, 2014, and 2016
• MySQL versions 5.5, 5.6, and 5.7
• Amazon Aurora (MySQL-compatible data source)
• Amazon Aurora (PostgreSQL-compatible data source)
• Amazon S3
• Amazon Redshift
• Amazon DynamoDB

Migration is a multiphase process
Phase Description Automation Effort (%)
1 Assessment SCT 2
2 Database Schema Conversion SCT/DMS 14
3 Application Conversion/Remediation SCT 25
4 Scripts Conversion SCT 7
5 Integration with third-party applications 3
6 Data Migration DMS 4
7 Functional testing of the entire system 29
8 Performance tuning SCT 2
9 Integration and deployment 7
10 Training and knowledge 2
11 Documentation and version control 2
12 Post-production support 3

Understand your environment
Your database
 What is the size of your database? Any large tables?
 How many schemas and tables do you have?
 What are the transaction boundaries?
 Unique data types to consider?
 Do you have LOBs in your tables, and how big are they?
 How hot is your source database?
And your network
 How to access your database (firewalls, tunnels, VPNs)
 Do you understand VPCs?
 Which security group should be used?
 Do you have enough bandwidth to move all your data?

What affects speed?
 The size of the source database and the associated activity
 The size of the target
 Availability of bandwidth
 Replication instance size
 Schema breakdown (a single large table can slow things down)
 LOBs in the schema
 Big transactions

Find the right people Start small Ramp up
Other considerations

Other best practices
• Keep backups disabled during the migration
• Set up parameter group for each instance to set
parameters per-instance
• Use AWS SCT to classify databases based on level of
work to convert

Other best practices for PostgreSQL
Use lowercase for DB object names
• Oracle by default saves everything in UPPERCASE
and PostgreSQL by default saves everything in
LOWERCASE
• Convert all the object names to lowercase in
PostgreSQL
• AWS SCT does this by default while migrating
objects from Oracle to PostgreSQL
• Lowercase names help to avoid object reference
complications

Other best practices for PostgreSQL
• Cutover planning
• Downtime window to stop writes on Oracle and
change configs to make PostgreSQL the new data
source
• Enable all user triggers in PostgreSQL after stopping
replication and before connecting users
• Pre-cutover validation steps
• Post-cutover validation steps
• Rollback strategy
• Use log_fdw, pg_repack, and pg_audit post-cutover

Resources available to customers - SCT
User Guide: Review technical docs at
aws.amazon.com/documentation/SchemaConversionTool/
or choose the Download button.
Download area: Get installation files for the
Schema Conversion Tool.
Support forums: Ask questions and review
how-to guides.
https://forums.aws.amazon.com/forum.jspa?forumID=208.

AWS Database Migration Service

Thank You!

Heterogenous Migration with DMS & SCT - Michael Russo

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Heterogenous Migration with DMS & SCT - Michael Russo

Similar to Heterogenous Migration with DMS & SCT - Michael Russo (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Heterogenous Migration with DMS & SCT - Michael Russo