Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Migrating Your NoSQL Database to Amazon DynamoDB (DAT314) - AWS re:Invent 2018

709 views

Published on

AWS Database Migration Service (AWS DMS) and AWS Schema Conversion Tool (AWS SCT) can help migrate databases from many supported data sources to supported targets. In this session, we review how the combination of AWS DMS and AWS SCT can help migrate your NoSQL databases, such as MongoDB and Cassandra, to Amazon DynamoDB. We provide an overview of AWS DMS and AWS SCT, and we demonstrate migrating a sample Cassandra database into DynamoDB.

  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Migrating Your NoSQL Database to Amazon DynamoDB (DAT314) - AWS re:Invent 2018

  1. 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migrating Your NoSQL Database to Amazon DynamoDB Arun Thiagarajan Product Manager AWS Database Migration Service/Schema Conversion Tool D A T 3 1 4
  2. 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda A quick overview of Amazon DynamoDB Why you should migrate from self-managed NoSQL databases An overview of AWS Database Migration Tool (AWS DMS) and AWS Schema Conversion Tool (AWS SCT) How to migrate from MongoDB and Cassandra to Amazon DynamoDB Demo Q&A
  3. 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  4. 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon DynamoDB Fully managed nonrelational database for any scale High performance • Fast, consistent performance • Virtually unlimited throughput • Virtually unlimited storage Enterprise ready • Encryption at rest and transit • Fine-grained access control • PCI, HIPAA, FIPS 140-2 eligible Fully managed • Maintenance-free • Serverless • Auto scaling • Backup and restore • Global tables
  5. 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fully managed Maintenance-free Security • Operating-system patching • Database patching • Access control • Audit • Encryption • Compliance Durability • Sustain server, rack, and data center outages • Re-replicate data quickly upon hardware failure • Manage backup and restore Availability • High-availability configuration • Monitoring • Cross-region replication Performance • Performance tuning • Indexing • In-memory caching Scalability • Capacity planning • Host provisioning • Host repair and retirement
  6. 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  7. 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why should you migrate from self managed NoSQL databases? With MongoDB/Cassandra, you deal with: • Outages (infrastructure failures, buggy patches) → Lost revenue • Lesser performance → Unpredictable user experience • Extra effort to scale, upgrade, and maintain → Lost time and resources • Required, specialized training → Less time to develop features that matter
  8. 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  9. 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What are AWS DMS and AWS SCT? AWS Schema Conversion Tool converts your commercial database and data warehouse schemas to open-source engines or AWS-native services, such as Amazon Aurora and Redshift AWS Database Migration Service easily and securely migrates and/or replicate your databases and data warehouses to AWS
  10. 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. >100,000 databases migrated with DMS
  11. 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Schema Conversion Tool • Migrate your data Migrate your Cassandra database using SCT Data Extractors Modernize • Modernize your database schema Convert your Cassandra database to Amazon DynamoDB Amazon DynamoDB
  12. 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS SCT product highlights Assess Plan Convert schema and code Optimize Migrate
  13. 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Database Migration Service Migrate • Migrate business-critical applications • Migrate data warehouse to Redshift • Upgrade to a minor version • Consolidate shards into Aurora • Archive old data • Migrate from NoSQL to SQL, SQL to NoSQL, or NoSQL to NoSQL Amazon RDS Amazon Redshift Amazon Aurora Amazon DynamoDB Amazon S3
  14. 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Database Migration Service Replicate • Create cross-region Read Replicas • Run your analytics in the cloud • Populate your data lake
  15. 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DMS—Endpoint support expansion Amazon Redshift Amazon DynamoDB Amazon S3 Amazon S3 Amazon Aurora Amazon Aurora Oracle SQL Server Netezza Greenplum Vertica Teradata AWS Snowball Edge MongoDB Cassandra Amazon Elasticsearch Amazon Kinesis
  16. 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DMS product highlights Verify Validate Ship with Snowball Secure
  17. 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customer premises AWS Internet VPN Migrating DB using AWS Snowball AWS DMS Local replication agent (through AWS SCT) AWS Snowball
  18. 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  19. 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Core Components MongoDB Amazon DynamoDB Collection Table Document Item Field Attribute Secondary Index Secondary Index
  20. 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migration Approaches Document mode: In this mode, AWS DMS migrates all the JSON data into a single column named “_doc” in the target DynamoDB table. Table mode: In this mode, AWS DMS scans a specified number of documents in the MongoDB database and creates a sample schema with all the keys and their types. During migration, you can use the object mapping feature in AWS DMS to transform the original data from MongoDB to the desired structure in DynamoDB.
  21. 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Object Mapping Example
  22. 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. MongoDB to Amazon DynamoDB Migration MongoDB Shard MongoDB Shard MongoDB Shard Amazon DynamoDB Reference: https://aws.amazon.com/blogs/database/performing-a-live- migration-from-a-mongodb-cluster-to-amazon-dynamodb/
  23. 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  24. 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cassandra to Amazon DynamoDB Migration Migrate data from Apache Cassandra and migrate to Amazon DynamoDB • Extracts data through local migration agents • Data is optimized for DynamoDB and saved in local files • Files are loaded to an Amazon S3 bucket (through network or Amazon Snowball) and then to Amazon DynamoDB AWS SCT S3 Bucket Amazon DynamoDB AWS DMS
  25. 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cassandra to DynamoDB migration—Step 1 Clone your Cassandra data center:
  26. 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cassandra to DynamoDB migration—Step 2 AWS SCTApache Cassandra Convert your schema from Cassandra to DynamoDB
  27. 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cassandra to DynamoDB migration—Step 3 Apache Cassandra AWS SCT Migrate your data from Cassandra to DynamoDB
  28. 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Cassandra to DynamoDB migration Step 1 Step 2 Step 3
  29. 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  30. 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Learn more about DynamoDB in the below sessions DAT201 – A Deep Dive into What’s New for Amazon DynamoDB DAT303 – Protecting Your Greatest Asset (Your Data): Security Best Practices on DynamoDB DAT311– Building Serverless Applications with Amazon DynamoDB & AWS Lambda – Workshop DAT314 – Migrating Your NoSQL Database to Amazon DynamoDB DAT320 – Becoming a Nimble Giant: How Amazon DynamoDB Serves Nike at Scale DAT321 – Amazon DynamoDB Under the Hood: How We Built the Hyper-Scale Database of the Future DAT325 – How Oath (a Verizon Company) Built a Multi-Region GDPR Application with Amazon DynamoDB DAT332 – Why GE Aviation Migrated from Cassandra to Amazon DynamoDB DAT333 – Real-World Use Cases for Amazon DynamoDB DAT341 – Migrating Financial and Accounting Systems from Oracle to Amazon DynamoDB DAT342 – Implementing Microservice Architectures with Amazon DynamoDB and AWS Lambda DAT345 – How GumGum Migrated from Cassandra to Amazon DynamoDB DAT347 – How Amazon Migrated Items and Offers for Retail, Marketplace, and Digital to DynamoDB DAT349 – Deep Dive on Amazon DynamoDB Global Tables DAT352 – Migrate Your Nonrelational Database to AWS DAT357 – Build Internet-Scale Apps with Amazon DynamoDB DAT365 – Separating Fact from Fiction: Amazon DynamoDB Scaling and Adaptive Capacity DAT401 – Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB DAT404 – Advanced Design Patterns for Amazon DynamoDB – Workshop
  31. 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. But wait, there’s more about DynamoDB • A list of all DynamoDB sessions, workshops, and chalk talks • Migrating Apache Cassandra databases to DynamoDB • What’s new with DynamoDB • Purpose-built databases in AWS • DynamoDB service level agreement • Adaptive capacity • Point-in-time recovery (PITR) • Global tables
  32. 32. Follow @dynamodb to get the latest
  33. 33. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Arun Thiagarajan
  34. 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

×