Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to choose the right database for your workload

199 views

Published on

How to choose the right database for your workload

  • Be the first to comment

  • Be the first to like this

How to choose the right database for your workload

  1. 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T How to choose the right database for your workload Boaz Ziniman Technical Evangelist Amazon Web Services D A T 0 0 1 Robert Jan Erkelens Program Manager Cloud 2020 Portbase Melvyn de Kort Cloud Architect Portbase @ziniman ziniman
  2. 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Why did you choose this database? “Because we heard X is the best new thing.” “Because we have a site license for X.” “Because X is what we know how to use.”
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Why should you choose this database? “Because this database is purpose built to support what my application is designed to do.”
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Modern apps have modern requirements Users: 1 million+ Data volume: TB–PB–EB Locality: Global Performance: Milliseconds–microseconds Request rate: Millions per second Access: Web, Mobile, IoT, Devices Scale: Up-down, Out-in Economics: Pay for what you use Developer access: No assembly requiredSocial mediaRide hailing Media streaming Dating
  5. 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Referential integrity, ACID transactions, schema- on-write Low-latency, key lookups with high throughput and fast ingestion of data Indexing and storing documents with support for query on any attribute Microseconds latency, key- based queries, and specialized data structures Creating and navigating data relations easily and quickly Lift and shift, EMR, CRM, finance Real-time bidding, shopping cart, social Content management, personalization, mobile Leaderboards, real-time analytics, caching Fraud detection, social networking, recommendation engine Indexing and searching semistructured logs and data Product catalog, help and FAQs, full text Collect, store, and process data sequenced by time IoT applications, event tracking Complete, immutable, and verifiable history of all changes Systems of record, supply chain, health care, registrations, financial Common data categories and use cases Relational Key-value Document In-memory Graph Search Time-series Ledger
  6. 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS: Purpose-built databases Relational Key-value Document In-memory Graph Search Amazon DynamoDB Amazon Neptune Amazon RDS Aurora CommercialCommunity Amazon ElastiCache Amazon Elasticsearch Service Amazon DocumentDB Time-series Ledger Amazon Timestream Amazon Quantum Ledger DatabaseMemcachedRedis
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CHALLENGE Wanted to enable anyone to learn a language for free. SOLUTION Purpose-built databases from AWS: • DynamoDB: 31B items tracking which language exercises completed • Aurora: primary transactional database for user data • ElastiCache: instant access to common words and phrases Result: More people learning a language on Duolingo than entire US school system 300M total users 7B exercises per month
  8. 8. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Relational Key-value Document Graph Let’s take a closer look at…
  9. 9. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Relational Key-value Document Graph Relational
  10. 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Patient * Patient ID First Name Last Name Gender DOB * Doctor ID Visit * Visit ID * Patient ID * Hospital ID Date * Treatment ID Medical Treatment * Treatment ID Procedure How Performed Adverse Outcome Contraindication Doctor * Doctor ID First Name Last Name Medical Specialty * Hospital Affiliation Hospital * Hospital ID Name Address Rating Relational Referential integrity, ACID transactions, schema-on-write Lift and shift, ERP, CRM, finance
  11. 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Relational // Doctors affiliated with Mercy hospital SELECT d.first_name, d.last_name FROM doctor as d, hospital as h WHERE d.hospital = h.hospital_id AND h.name = ‘Mercy'; // Number of patient visits each doctor completed last week SELECT d.first_name, d.last_name, count(*) FROM visit as v, hospital as h, doctor as d WHERE v.hospital_id = h.hospital_id AND h.hospital_id = d.hospital AND v.t_date > date_trunc('week’, CURRENT_TIMESTAMP - interval '1 week') GROUP BY d.first_name, d.last_name; Patient * Patient ID First Name Last Name Gender DOB * Doctor ID Visit * Visit ID * Patient ID * Hospital ID Date * Treatment ID Medical Treatment * Treatment ID Procedure How Performed Adverse Outcome Contraindication Doctor * Doctor ID First Name Last Name Medical Specialty * Hospital Affiliation Hospital * Hospital ID Name Address Rating
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon Relational Database Service (RDS) Managed relational database service with a choice of six popular database engines Easy to administer Available and durable Highly scalable Fast and secure No need for infrastructure provisioning, installing, and maintaining DB software Automatic Multi-AZ data replication; automated backup, snapshots, failover Scale database compute and storage with a few clicks with no app downtime SSD storage and guaranteed provisioned I/O; data encryption at rest and in transit
  13. 13. The right database for Portbase Melvyn de Kort Cloud Architect @ Portbase Robert Jan Erkelens Program Manager Cloud 2020 @ Portbase
  14. 14. 4.000 parties 14.500 users 43 services 90.000.000 messages/yr
  15. 15. Growing Demand
  16. 16. DevOps and Public Cloud Strategy
  17. 17. Our Database is a Mission Critical Component !!!
  18. 18. Database requirements – migration Oracle Relational Database Existing Oracle license Applications use Oracle specific database features Low risk migration Low latency communication with database
  19. 19. Database requirements – production High availability Managed maintenance and upgrades In depth monitoring Tech support on demand
  20. 20. Database requirements – strategy Being inside AWS ecosystem Flexibility to migrate to application specific database solutions, e.g. DynamoDB ElasticSearch
  21. 21. Amazon RDS for Oracle Amazon RDS Multi-AZ deployment Amazon RDS CloudWatch Metrics Fully managed database service Encryption at rest & in transit
  22. 22. Amazon RDS for Oracle – extras Amazon RDS Performance Insights AWS Glue for Business Intelligence Amazon RDS Read Replica for Oracle Amazon S3 integration for data ingress and egress Amazon DMS (Data Migration Services)
  23. 23. What has AWS RDS brought us? Results Business Value Monitoring and alerting are top notch Higher availability DevOps team is in full control Faster incident resolution Vertical scaling within minutes Prepared for changing demand Encryption (at rest & in transit) Enhanced security
  24. 24. Next steps Read replica for Reporting queries and copying data to Data Lake Data archival to S3 triggered by AWS Lambda Split the single big database into multiple smaller ones Move from Oracle to Aurora PostgreSQL compatible Move to Aurora Serverless
  25. 25. Thank you!
  26. 26. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Relational Key-value Document Graph Key-value & Document
  27. 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon DynamoDB Fast and flexible key value database service for any scale Comprehensive security Encrypts all data by default and fully integrates with AWS Identity and Access Management for robust security Performance at scale Consistent, single-digit millisecond response times at any scale; build applications with virtually unlimited throughput Global database for global users and apps Build global applications with fast access to local data by easily replicating tables across multiple AWS Regions Serverless No server provisioning, software patching, or upgrades; scales up or down automatically; continuously backs up your data
  28. 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon DynamoDB Data Structure Table Items Attributes Partition Key Sort Key Mandatory Key-value access pattern Determines data distribution Optional Model 1:N relationships Enables rich query capabilities All items for key ==, <, >, >=, <= “begins with” “between” “contains” “in” sorted results counts top/bottom N values
  29. 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Gamers Primary Key Attributes Gamer Tag Type Hammer57 Rank Level Points Tier 87 4050 Elite Status Health Progress 90 30 Weapon Class Damage Range Taser 87% 50 FluffyDuffy Rank Level Points Tier 5 1072 Trainee Status Health Progress 37 8 Key-value // Status of Hammer57 GET { TableName:"Gamers", Key: { "GamerTag":"Hammer57", "Type":"Status” } } // Return all Hammer57 Gamers GamerTag = :a :a Hammer57
  30. 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Document (client) (app) (database) JSON !JSON != Indexing and storing documents with support for query on any attribute Content management, personalization, mobile
  31. 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Document User profiles { id: 181276, username: "sue1942", name: {first: "Susan", last: "Benoit"} } { id: 181276, username: "sue1942", name: {first: "Susan", last: "Benoit"}, ExplodingAcorns: { hi_score: 3185400, global_rank: 5139, bonus_levels: true }, promotions: ["new user","5%",”acorn"] } Indexing and storing documents with support for query on any attribute Content management, personalization, mobile
  32. 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon DocumentDB Fast, scalable, and fully managed MongoDB-compatible database service Fast Scalable Fully managed MongoDB compatible Millions of requests per second with millisecond latency; twice the throughput of MongoDB Separation of compute and storage enables both layers to scale independently; scale out to 15 read replicas in minutes Managed by AWS: no hardware provisioning; auto patching, quick setup, secure, and automatic backups Compatible with MongoDB 3.6; use the same SDKs, tools, and applications with Amazon DocumentDB
  33. 33. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Relational Key-value Document Graph Graph
  34. 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Relationships enable new applications Retail fraud detectionRestaurant recommendationsSocial networks
  35. 35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Graph // Product recommendation to a user gremlin> V().has(‘name’,’sara’).as(‘customer’).out(‘follows’).in(‘follows’).out(‘purchased’) ( (‘customer’)).dedup() (‘name’) ('name') KNO W S PURCHASED PURCHASED FOLLOWS PURCHASED KNOWS PRODUCT SPORT FOLLOWS FOLLOWS // Identify a friend in common and make a recommendation gremlin> g.V().has('name','mary').as(‘start’). both('knows').both('knows’). where(neq(‘start’)). dedup().by('name').properties('name')
  36. 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Different approaches for highly connected data Purpose-built for a business process Purpose-built to answer questions about relationships
  37. 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon Neptune Fully managed graph database Easy Build powerful queries easily with Gremlin and SPARQL Fast Query billions of relationships with millisecond latency Open Supports Apache TinkerPop & W3C RDF graph models Reliable Six replicas of your data across three AZs with full backup and restore
  38. 38. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. https://github.com/aws-samples/aws-bookstore-demo-app
  39. 39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Bookstore Demo
  40. 40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Bookstore Demo
  41. 41. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  42. 42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon Timestream (sign up for the preview) Fast, scalable, fully managed time-series database 1,000x faster and 1/10th the cost of relational databases Collect data at the rate of millions of inserts per second (10M/second) Trillions of daily events Adaptive query processing engine maintains steady, predictable performance Time-series analytics Built-in functions for interpolation, smoothing, and approximation Serverless Automated setup, configuration, server provisioning, software patching
  43. 43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon Quantum Ledger Database (QLDB) (Preview) Fully managed ledger database Track and verify history of all changes made to your application’s data Immutable Maintains a sequenced record of all changes to your data, which cannot be deleted or modified; you have the ability to query and analyze the full history Cryptographically verifiable Uses cryptography to generate a secure output file of your data’s history Easy to use Easy to use, letting you use familiar database capabilities like SQL APIs for querying the data Highly scalable Executes 2–3X as many transactions than ledgers in common blockchain frameworks
  44. 44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What have we learned? Choose the right tool for the job Start from the application and work backwards Modern applications considerations Demo Application On GitHub: /aws-samples/aws- bookstore-demo-app Keep on Building ! AWS: Purpose-built databases Relational Key-value Document In-memory Graph Search Amazon DynamoDB Amazon Neptune Amazon RDS Aurora CommercialCommunity Amazon ElastiCache Amazon Elasticsearch Service Amazon DocumentDB Time-series Ledger Amazon Timestream Amazon Quantum Ledger DatabaseMemcachedRedis Social mediaRide hailing Media streaming Dating
  45. 45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Boaz Ziniman Technical Evangelist - Amazon Web Service @ziniman ziniman

×