Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Migrate a relational database to Aurora - ADB302 - Atlanta AWS Summit

174 views

Published on

In this hands-on lab, you use AWS Schema Conversion Tool and AWS Database Migration Service to migrate a relational database to Amazon Aurora PostgreSQL. Bring a laptop that has the Firefox or Chrome browser installed and a working AWS account. We provide an AWS CloudFormation template to configure the lab environment and help you walk through the migration process. Please be sure to bring your laptop.

  • Be the first to comment

  • Be the first to like this

Migrate a relational database to Aurora - ADB302 - Atlanta AWS Summit

  1. 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Database Freedom Kathy Gibbs Sr Specialist Solution Architect, Database AWS A D B 3 0 2
  2. 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Agenda Database Freedom Program Overview Migration Tools: • Database Migration Service (DMS) • Schema Conversion Tool (SCT) Lab: • Oracle / SQL Server to Amazon Aurora
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T https://www.theregister.co.uk/2017/01/30/oracle_effectively_doubles_licence_fees_to_run_in_aws/ https://www.infoworld.com/article/3164517/cloud-computing/oracle-is-pricing-itself-out-of-amazons-cloud.html
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What we hear from customers about legacy databases... AUDIT Very expensive Proprietary Lock-in Punitive licensing You’ve got mail
  5. 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Database Freedom is an AWS database migration initiative focused on accelerating enterprise migrations from commercial database engines (Oracle and SQL Server) to AWS native database services.
  6. 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Customers are migrating their workloads to AWS Verizon is migrating over 1,000 business-critical applications and database backend systems to AWS, several of which also include the migration of production databases to Amazon Aurora. By migrating from Microsoft SQL Server to Amazon Aurora, Ryanair can run one of the largest email campaigns in Europe with higher performance at a fraction of the cost, sending out 22 million emails daily to customers. Trimble migrated their Oracle databases to Amazon RDS and project they will pay about 1/4th of what they paid when managing their private infrastructure. Intuit migrated from Microsoft SQL Server to Amazon Redshift to reduce data-processing timelines and get insights to decision makers faster and more frequently. Equinox Fitness migrated its Teradata on-premises data warehouse to Amazon Redshift. They went from static reports to a modern data lake that delivers dynamic reports. Migrated their Market Data system from SQL Server to Aurora MySQL using AWS Database Migration Service(DMS) to replicate data nightly. Reduces their processing times from 8 hours to 3 hours. By December 2018, Amazon.com will have migrated 88% of their Oracle DBs (and 97% of critical system DBs) moved to Amazon Aurora and Amazon DynamoDB. They also migrated their 50 PB Oracle Data Warehouse to AWS (Amazon S3, Amazon Redshift, and Amazon EMR). Samsung Electronics migrated their Cassandra clusters to Amazon DynamoDB for their Samsung Cloud workload with 70% cost savings.
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Programs Discover Database Freedom with AWS Innovation • Tools: Database Migration Service (DMS) and Schema Conversion Tool (SCT) • AWS native managed database services • Optimized and new EC2 and RDS instance types • ProServe, Partners, Service Teams • Workload Qualification Framework • Patterns and Recommendations Expertise • AWS Professional Services, Partners, Product Teams • Migration Playbooks • Patterns and Recommendations • Proof-of-Concepts • Workshops • Incentives / Credits Database Freedom is an AWS Database Migration initiative focused on accelerating enterprise migrations from commercial database engines (Oracle and SQL Server) to AWS native database services.
  8. 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS migration options AWS DMS Database migration and replication managed service Benefit • Easily and securely migrate and/or replicate your databases and data warehouses to AWS • Migrate between different database engines • Low cost and global availability AWS SCT Development tooling to convert schemas between databases and data warehouses Benefit • Automates schema conversion including database structure and code • Minimizes manual effort of performing a schema re- write • Allows conversion from commercial to open source platforms Native Tooling Migration options included with the engine Benefit • Leverage a familiar environment • Full support for native features Open Source Ora2Pg and others Benefit • Tailored solution for a specific problem • Multiple tools can offer a more complete solution than any one product • Free Commercial Attunity, Golden Gate and others Benefit • Commercial solutions are available for a wide range of migration sources and targets including legacy mainframe systems
  9. 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Workload Qualification Framework (WQF) • The WQF includes self-service tools, a questionnaire, and design review guidance that Solutions Architects, Partners and Consultants can use to classifying workloads to determine the ease of migration, staff-hour consumption and to which appropriate AWS Service to migrate • There are two main types of database workloads: • Online Transaction Processing (OLTP): many small/fast transactions and queries via indexes that return few rows • Data Warehouse (DW): interactive and long-running queries that scan millions of rows and batched updates • WQF classifies OLTP and DW workloads into five categories: Category 1 ODBC/JBDC workloads Candidate for Re-Factor Category 2 Light, proprietary feature workloads Candidate for Re-Factor Category 3 Heavy, proprietary feature workloads Candidate for Re-Factor/Re-Platform Category 4 Engine-specific workloads Candidate for Re-Platform/Re-Host Category 5 Non-portable, high-risk or lift-and-shift Candidate for Re-Host
  10. 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Database Workload Analysis Disparate DB Workloads Oracle EBS PSFT / JDE / SAP Siebel / CRM DW / Lakes Custom Apps on Oracle DB SQL*Server / Other DB’s Big Data / NoSQL Outcomes Re-Factor Gathering Data and Organizing “Discovery & Planning” Workload Qualification Framework Assessment Questionnaires 2-Day Workshops • Application Discovery RDS-Aurora MySQL RDS-Aurora PostgreSQL DynamoDB Redshift/Others Re-Platform RDS-Oracle RDS-SQL Server RDS-MySQL RDS-PostgreSQL Re-Host Elastic Compute (EC2) Elastic Block Storage (EBS) Other Options Simple Storage (S3) • Migration Hub • Migration Readiness Program IncreasingValue
  11. 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Legacy Databases to AWS Migration Playbooks • Topic-by-topic overview of how to migrate databases and data warehouses to AWS services • Covers all proprietary features and the different database objects • Migration best practices • Oracle to Aurora PostgreSQL – available • SQL Server to Aurora MySQL – available • SQL Server to Aurora PostgreSQL – available • Oracle to Aurora MySQL, Cassandra to DynamoDB, Data warehouses to Redshift, and Db2 LUW to Aurora PostgreSQL – Q1 SCT DMS Playbook Schema Data Best Practices
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS database partners Migration SI Partners License Advisory Partners
  13. 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Database migration resources Migration Methodology AWS Tools and Specialist Teams AWS Partners AWS Investment AWS Training AWS Professional Services
  14. 14. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  15. 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What are DMS and SCT? Our goal: Allow customers the freedom to choose the best data platform for their needs #DBFreedom AWS Schema Conversion Tool (SCT) converts your commercial database and data warehouse schemas to open-source engines or AWS-native services, such as Amazon Aurora and Redshift AWS Database Migration Service (DMS) easily and securely migrates and/or replicate your databases and data warehouses to AWS
  16. 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T When to use DMS and SCT? Modernize Migrate Replicate
  17. 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS Schema Conversion Tool (SCT) • Modernize your data warehouse Convert your Oracle, SQL Server, Netezza, Greenplum, Vertica or Teradata to Amazon Redshift Modernize Amazon Aurora Amazon Redshift • Modernize your database Convert your Oracle, SQL Server, or Db2 LUW to PostgreSQL, MySQL, or Amazon Aurora MySQL PostgreSQL
  18. 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS Database Migration Service (DMS) Migrate • Migrate business-critical applications • Migrate data warehouse to Redshift • Upgrade to a minor version • Consolidate shards into Aurora • Archive old data • Migrate from NoSQL to SQL, SQL to NoSQL or NoSQL to NoSQL Amazon RDS Amazon Redshift Amazon Aurora Amazon DynamoDB Amazon S3
  19. 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS Database Migration Service (DMS) Replicate • Create cross region Read Replicas • Run your analytics in the cloud • Populate your data lake
  20. 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T DMS Momentum – Automation and Acceleration
  21. 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS Schema Conversion Tool
  22. 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS SCT Product Highlights • Assessment of migration compatibility of source databases with open-source database engines – RDS for MySQL, RDS for PostgreSQL, and Amazon Aurora • Recommends best target engine • Provides details level of efforts to complete migration Assessment Report • Attempts to convert all schema and code objects to the target engine, including stored procedures and functions • Scans and converts embedded SQL statements in app code • Generates a report with recommendations Converts Schema and Code • Extracts data through local migration agents • Files are loaded to an Amazon S3 bucket and to Amazon Redshift • Netezza • Vertica • Greenplum • Teradata • Oracle • SQL Server Extracts and Migrates DW to Amazon Redshift
  23. 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T SCT helps with converting tables, views, and code Sequences User-defined types Synonyms Packages Stored procedures Functions Triggers Schemas Tables Indexes Views Sort and distribution keys
  24. 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T SCT Migration Assessment Report • Assessment of migration compatibility of source databases with open-source database engines – RDS MySQL, RDS PostgreSQL and Aurora • Recommends best target engine • Provides details level of efforts to complete migration
  25. 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T When to use SCT? Modernize & Migrate Modernize your database tier Modernize and Migrate Cassandra to Amazon DynamoDB Amazon Aurora Amazon Redshift Modernize and Migrate your Data Warehouse to Amazon Redshift Amazon DynamoDB
  26. 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS Database Migration Service
  27. 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS DMS Product Highlights • Checks migration task settings prior to launch • Alerts for potential issues, such as unsupported data types • Prevent unnecessary, time consuming runtime migration failures Pre-migration Assessment • Validates that all data selected for migration migrated properly • Includes both stages of full load and CDC • Applies for both homogeneous and heterogeneous migrations Data Validation Use AWS Snowball & AWS DMS to migrate data to AWS • Migrate large databases (over 5TB) • Migrate many databases at once • Migrate over slow network • Push vs. Pull Snowball Integration
  28. 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Migration Validation Pre-migration Assessment • Checks migration task settings prior to launch • Alerts for potential issues, such as unsupported data types Post-migration Assessment • Validates that all data selected for migration migrated properly
  29. 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T When to use DMS? Amazon RedshiftAmazon DynamoDB Amazon S3 Amazon S3 Amazon Aurora Amazon Aurora Oracle SQL Server Netezza Greenplum Vertica Teradata AWS Snowball Edge MongoDB Cassandra Amazon Elasticsearch Amazon Kinesis
  30. 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T How does it work?
  31. 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Database migration process Step 1: Convert or Copy your Schema Source DB or DW AWS SCT Native Tool Destination DB or DW Step 2: Move your data Source DB or DW AWS SCT Destination DB or DW AWS DMS
  32. 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Customer premises Application users AWS Internet VPN Start a replication instance Connect to source and target databases Select tables, schemas, or databases  Let AWS DMS create tables, load data, and keep them in sync  Switch applications over to the target at your convenience Keep your apps running during the migration AWS DMS
  33. 33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Multi-AZ option for high availability Customer premises or AWS AWS Internet VPN AWS DMS AWS DMS
  34. 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Load is table by table Replication instance Source Target
  35. 35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Change data capture (CDC) and apply Replication instanceSource Target Update t1 t2 t1 t2 Transactions Change apply after bulk load
  36. 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What else can I do?
  37. 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Homogenous or heterogeneous SQL Server MySQL Oracle Oracle Oracle S3
  38. 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Source Take it all—or not Target Replication instance
  39. 39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Source Target Consolidation Source Source Replication instance
  40. 40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Partition Target Source Target Target Replication instance
  41. 41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Other Migrations
  42. 42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Data Warehouse to Redshift Migration Extract Data from your data warehouse and migrate to Amazon Redshift • Data is extracted through local migration agents • Data is optimized for Redshift and saved in local files • Files are loaded to an Amazon S3 bucket (through network or Amazon Snowball) and then to Amazon Redshift Amazon RedshiftAWS SCT S3 Bucket
  43. 43. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  44. 44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Getting started Determine Database Migration Path Project Based or Large Scale Re-Factoring 1 3 Use DMS and SCT tools to kick-off database migration pilot! 2 Initiate database migration discovery and build business case https://github.com/katgibbs/atlsummit
  45. 45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Resources Database Migrations https://aws.amazon.com/solutions/database-migrations/ Database Freedom https://aws.amazon.com/solutions/databasemigrations/database-freedom/ DMS resources https://aws.amazon.com/dms/resources/ Contact: Sandeep Brahmarouthu – BDM Database Freedom brahmaro@amazon.com Angela Wyant – BDM PostgreSQL aewyant@amazon.com Jennifer Chang– BDM MySQL changjen@amazon.com
  46. 46. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  47. 47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

×