SlideShare a Scribd company logo
1 of 21
© 2015 CLUSTRIX
The First Scale-out SQL Database Engineered for Today’s Cloud
Beyond Aurora.
Scale-out SQL databases for AWS
Magnus Data
SCALE OUT RDBMS
3/4/2016
MagnusData
Agenda
 Market Landscape of DB market
 Options to Scale DB
 Scale-Out Architecture
 Comparisons of solutions for high transaction relational databases
3/4/2016
Generalized and Specialized
3/4/2016
High Concurrency/Write heavy /Real Time Analytics Historical Analytics Exploratory
Transactional Analytics
Traditional Databases
No
SQL
DW/Analytical
DBMS
Operational
System/OLTP (New
SQL)
Hadoop
Scale-Up vs. Scale-Out
3/4/2016
Scale-Out databases
Transactions Per Second
LatencyHigh
High
Scale-Up Databases
(like Aurora
and MySQL)
RDBMS Scaling Techniques
 Scale-Up
 Master Slave
 Master Master
 MySQL Clustering Technologies
 Sharding
 Scale-Out
3/4/2016
Options to Scale DBMS
3/4/2016
DBMS
Scale Out
e.g., MongoDB
No transactions
May have weak consistency
(CAP)
Application involves DB
Coding
e.g. ClustrixDB
ACID
Proven Scalability
(Reads and Writes)
Shared Nothing
Scale Up
e.g., Aurora
Reads Scale
limited scalability on writes
Not Shared nothing scale
out
Scaling-Up
 Keep increasing the size of the (single) database server
 Pros
 Simple, no application changes needed
 Cons
 Expensive. At some point, you’re paying 5x for 2x the performance
 ‘Exotic’ hardware (128 cores and above) become price prohibitive
 Eventually you ‘hit the wall’, and you literally cannot scale-up anymore
8
Scaling Reads: Master/Slave
 Add a ‘Slave’ read-server(s) to your ‘Master’ database server
 Pros
 Reasonably simple to implement.
 Read/write fan-out can be done at the proxy level
 Cons
 Only adds Read performance
 Data consistency issues can occur, especially if the application isn’t coded to
ensure reads from the slave are consistent with reads from the master
9
Scaling Writes: Master/Master
10
 Add additional ‘Master’(s) to your ‘Master’ database server
 Pros
 Adds Write scaling without needing to shard
 Cons
 Adds write scaling at the cost of read-slaves
 Adding read-slaves would add even more latency
 Application changes are required to ensure data consistency / conflict resolution
Scaling Reads & Writes: Sharding
11
SHARDO1 SHARDO2 SHARDO3 SHARDO4
 Partitioning tables across separate database servers
 Pros
 Adds both write and read scaling
 Cons
 Loses the ability of an RDBMS to manage transactionality, referential integrity and ACID
 ACID compliance & transactionality must be managed at the application level
 Consistent backups across all the shards are very hard to manage
 Read and Writes can be skewed / unbalanced
 Application changes can be significant
A - K L - O P - S T - Z
Scaling Reads & Writes: MySQL Cluster
 Provides shared-nothing clustering and auto-sharding for MySQL. (designed for Telco deployments: minimal cross-node
transactions, HA emphasis)
 Pros
 Distributed, multi-master model
 Provides high availability and high throughput
 Cons
 Only supports read-committed isolation
 Long-running transactions can block a node restart
 SBR replication not supported
 Range scans are expensive and lower performance than MySQL
 Unclear how it scales with many nodes
12
Application Workload Partitioning
13
 Partition entire application + RDBMS stack across several “pods”
 Pros
 Adds both write and read scaling
 Flexible: can keep scaling with addition of pods
 Cons
 No data consistency across pods (only suited for cases where it is not
needed)
 High overhead in DBMS maintenance and upgrade
 Queries / Reports across all pods can be very complex
 Complex environment to setup and support
APP
APP
APP
APP
APP
APP
DBMS Capacity, Elasticity and Resiliency
14
Scale-up
Master – Slave
Master – Master
MySQL Cluster
Sharding
Scale-Out
DBMS Scaling
Many cores – very expensive
Reads Only
Read / Write
Read / Write
Unbalanced Read/Writes
Read / Write
Capacity
Single Point Failure
Fail-over
Yes
Yes
Multiple points of failure
Yes
ResiliencyElasticity
No
No
No
No
No
Yes
None
Yes – for read scale
High – update conflict
None (or minor)
Very High
None
Application Impact
DBMS Architecture-Scale out
3/4/2016
Shared Nothing Architecture
Compiler Map
Engine Data
Compiler Map
Engine Data
Compiler Map
Engine Data
Each Node Contains:
 Query Parser/Planner: distribute partial query
fragments to the nodes.
 Data Map: all nodes metadata about data
across the cluster
 Database Engine: all nodes can perform all
database operations (no leader, aggregator,
leaf, data-only, etc nodes)
 Data: Table Distributed: All table auto-
redistributed
BillionsofRows
Database
Tables
S1 S2
S2
S3
S3
S4
S4
S5
S5
Intelligent Data Distribution
16
S1
ClustrixDB
 Tables Auto Distributed across nodes
 Tunable amount of redundancy of data across nodes
 Tables are auto distributed, auto-protected
Query
Distributed Query Processing
17
ClustrixDB
Load
Balancer
TRXTRXTRX
 Queries are fielded by any peer node
 Routed to node holding the data
 Complex queries are split into steps and processed in parallel
 Automatically distributed for optimized performance
 All nodes handle writes and reads
 Result is aggregated and returned to the user
DBMS Capacity, Elasticity and Resiliency
18
Features ClustrixDB Aurora
Write Scalability Writes scales by adding nodes Cannot add write nodes
High Concurrency Latency Low with High concurrency Latency climbs quickly with high
concurrency
ACID Yes Yes
On-Demand Write Scale Yes No
Automatically Distributed
queries
Yes: No Application changes No: Read/Write fanout needed.
Write contention on Master
Cloud/On Premises Yes No, only AWS Cloud
Shared Nothing Storage Yes: Parallel data access No: Contention at high write
concurrency
Benchmark Results
3/4/2016
0
10
20
30
0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000
AverageLatency(ms)
Throughput (tps)
Sysbench OLTP 90:10 Mix
0
10
20
30
0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000
AverageLatency(ms)
Throughput (tps)
Sysbench OLTP 90:10 Mix
Clustrix 4 Node Aurora Mysql RDS
Scalability Test
3/4/2016
0
10
20
30
0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000
AverageLatency(ms)
Throughput (tps)
Sysbench OLTP 90:10 Mix
21
Thank you.
Q&A

More Related Content

What's hot

Workshop - How to benchmark your database
Workshop - How to benchmark your databaseWorkshop - How to benchmark your database
Workshop - How to benchmark your databaseScyllaDB
 
How Alibaba Cloud scaled ApsaraDB with MariaDB MaxScale
How Alibaba Cloud scaled ApsaraDB with MariaDB MaxScaleHow Alibaba Cloud scaled ApsaraDB with MariaDB MaxScale
How Alibaba Cloud scaled ApsaraDB with MariaDB MaxScaleMariaDB plc
 
Infosys Ltd: Performance Tuning - A Key to Successful Cassandra Migration
Infosys Ltd: Performance Tuning - A Key to Successful Cassandra MigrationInfosys Ltd: Performance Tuning - A Key to Successful Cassandra Migration
Infosys Ltd: Performance Tuning - A Key to Successful Cassandra MigrationDataStax Academy
 
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...DataStax
 
Cassandra at eBay - Cassandra Summit 2013
Cassandra at eBay - Cassandra Summit 2013Cassandra at eBay - Cassandra Summit 2013
Cassandra at eBay - Cassandra Summit 2013Jay Patel
 
The Future of Postgres Sharding / Bruce Momjian (PostgreSQL)
The Future of Postgres Sharding / Bruce Momjian (PostgreSQL)The Future of Postgres Sharding / Bruce Momjian (PostgreSQL)
The Future of Postgres Sharding / Bruce Momjian (PostgreSQL)Ontico
 
Cassandra implementation for collecting data and presenting data
Cassandra implementation for collecting data and presenting dataCassandra implementation for collecting data and presenting data
Cassandra implementation for collecting data and presenting dataChen Robert
 
How to power microservices with MariaDB
How to power microservices with MariaDBHow to power microservices with MariaDB
How to power microservices with MariaDBMariaDB plc
 
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...Amazon Web Services
 
Scylla Summit 2018: Scylla Feature Talks - SSTables 3.0 File Format
Scylla Summit 2018: Scylla Feature Talks - SSTables 3.0 File FormatScylla Summit 2018: Scylla Feature Talks - SSTables 3.0 File Format
Scylla Summit 2018: Scylla Feature Talks - SSTables 3.0 File FormatScyllaDB
 
Webinar: DataStax Training - Everything you need to become a Cassandra Rockstar
Webinar: DataStax Training - Everything you need to become a Cassandra RockstarWebinar: DataStax Training - Everything you need to become a Cassandra Rockstar
Webinar: DataStax Training - Everything you need to become a Cassandra RockstarDataStax
 
Scylla Summit 2018: Adventures in AdTech: Processing 50 Billion User Profiles...
Scylla Summit 2018: Adventures in AdTech: Processing 50 Billion User Profiles...Scylla Summit 2018: Adventures in AdTech: Processing 50 Billion User Profiles...
Scylla Summit 2018: Adventures in AdTech: Processing 50 Billion User Profiles...ScyllaDB
 
Shift: Real World Migration from MongoDB to Cassandra
Shift: Real World Migration from MongoDB to CassandraShift: Real World Migration from MongoDB to Cassandra
Shift: Real World Migration from MongoDB to CassandraDataStax
 
Cloudian HyperStore 'Forever Live' Storage Platform
Cloudian HyperStore 'Forever Live' Storage PlatformCloudian HyperStore 'Forever Live' Storage Platform
Cloudian HyperStore 'Forever Live' Storage PlatformCloudian
 
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life InsuranceHBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life InsuranceMichael Stack
 
Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...
Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...
Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...ScyllaDB
 
Develop Scalable Applications with DataStax Drivers (Alex Popescu, Bulat Shak...
Develop Scalable Applications with DataStax Drivers (Alex Popescu, Bulat Shak...Develop Scalable Applications with DataStax Drivers (Alex Popescu, Bulat Shak...
Develop Scalable Applications with DataStax Drivers (Alex Popescu, Bulat Shak...DataStax
 
Transform your DBMS to drive engagement innovation with Big Data
Transform your DBMS to drive engagement innovation with Big DataTransform your DBMS to drive engagement innovation with Big Data
Transform your DBMS to drive engagement innovation with Big DataAshnikbiz
 
What is Change Data Capture (CDC) and Why is it Important?
What is Change Data Capture (CDC) and Why is it Important?What is Change Data Capture (CDC) and Why is it Important?
What is Change Data Capture (CDC) and Why is it Important?FlyData Inc.
 

What's hot (20)

Workshop - How to benchmark your database
Workshop - How to benchmark your databaseWorkshop - How to benchmark your database
Workshop - How to benchmark your database
 
How Alibaba Cloud scaled ApsaraDB with MariaDB MaxScale
How Alibaba Cloud scaled ApsaraDB with MariaDB MaxScaleHow Alibaba Cloud scaled ApsaraDB with MariaDB MaxScale
How Alibaba Cloud scaled ApsaraDB with MariaDB MaxScale
 
Infosys Ltd: Performance Tuning - A Key to Successful Cassandra Migration
Infosys Ltd: Performance Tuning - A Key to Successful Cassandra MigrationInfosys Ltd: Performance Tuning - A Key to Successful Cassandra Migration
Infosys Ltd: Performance Tuning - A Key to Successful Cassandra Migration
 
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
 
Cassandra at eBay - Cassandra Summit 2013
Cassandra at eBay - Cassandra Summit 2013Cassandra at eBay - Cassandra Summit 2013
Cassandra at eBay - Cassandra Summit 2013
 
The Future of Postgres Sharding / Bruce Momjian (PostgreSQL)
The Future of Postgres Sharding / Bruce Momjian (PostgreSQL)The Future of Postgres Sharding / Bruce Momjian (PostgreSQL)
The Future of Postgres Sharding / Bruce Momjian (PostgreSQL)
 
Cassandra implementation for collecting data and presenting data
Cassandra implementation for collecting data and presenting dataCassandra implementation for collecting data and presenting data
Cassandra implementation for collecting data and presenting data
 
How to power microservices with MariaDB
How to power microservices with MariaDBHow to power microservices with MariaDB
How to power microservices with MariaDB
 
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...
 
Scylla Summit 2018: Scylla Feature Talks - SSTables 3.0 File Format
Scylla Summit 2018: Scylla Feature Talks - SSTables 3.0 File FormatScylla Summit 2018: Scylla Feature Talks - SSTables 3.0 File Format
Scylla Summit 2018: Scylla Feature Talks - SSTables 3.0 File Format
 
Webinar: DataStax Training - Everything you need to become a Cassandra Rockstar
Webinar: DataStax Training - Everything you need to become a Cassandra RockstarWebinar: DataStax Training - Everything you need to become a Cassandra Rockstar
Webinar: DataStax Training - Everything you need to become a Cassandra Rockstar
 
Scylla Summit 2018: Adventures in AdTech: Processing 50 Billion User Profiles...
Scylla Summit 2018: Adventures in AdTech: Processing 50 Billion User Profiles...Scylla Summit 2018: Adventures in AdTech: Processing 50 Billion User Profiles...
Scylla Summit 2018: Adventures in AdTech: Processing 50 Billion User Profiles...
 
Shift: Real World Migration from MongoDB to Cassandra
Shift: Real World Migration from MongoDB to CassandraShift: Real World Migration from MongoDB to Cassandra
Shift: Real World Migration from MongoDB to Cassandra
 
Cloudian HyperStore 'Forever Live' Storage Platform
Cloudian HyperStore 'Forever Live' Storage PlatformCloudian HyperStore 'Forever Live' Storage Platform
Cloudian HyperStore 'Forever Live' Storage Platform
 
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life InsuranceHBaseConAsia2018 Track3-3: HBase at China Life Insurance
HBaseConAsia2018 Track3-3: HBase at China Life Insurance
 
Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...
Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...
Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...
 
Voldemort
VoldemortVoldemort
Voldemort
 
Develop Scalable Applications with DataStax Drivers (Alex Popescu, Bulat Shak...
Develop Scalable Applications with DataStax Drivers (Alex Popescu, Bulat Shak...Develop Scalable Applications with DataStax Drivers (Alex Popescu, Bulat Shak...
Develop Scalable Applications with DataStax Drivers (Alex Popescu, Bulat Shak...
 
Transform your DBMS to drive engagement innovation with Big Data
Transform your DBMS to drive engagement innovation with Big DataTransform your DBMS to drive engagement innovation with Big Data
Transform your DBMS to drive engagement innovation with Big Data
 
What is Change Data Capture (CDC) and Why is it Important?
What is Change Data Capture (CDC) and Why is it Important?What is Change Data Capture (CDC) and Why is it Important?
What is Change Data Capture (CDC) and Why is it Important?
 

Similar to Beyond Aurora. Scale-out SQL databases for AWS

Scaling RDBMS on AWS- ClustrixDB @AWS Meetup 20160711
Scaling RDBMS on AWS- ClustrixDB @AWS Meetup 20160711Scaling RDBMS on AWS- ClustrixDB @AWS Meetup 20160711
Scaling RDBMS on AWS- ClustrixDB @AWS Meetup 20160711Dave Anselmi
 
NOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQLNOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQLRamakant Soni
 
NOSQL in big data is the not only structure langua.pdf
NOSQL in big data is the not only structure langua.pdfNOSQL in big data is the not only structure langua.pdf
NOSQL in big data is the not only structure langua.pdfajajkhan16
 
NoSQL Options Compared
NoSQL Options ComparedNoSQL Options Compared
NoSQL Options ComparedSergey Bushik
 
Getting started with Amazon Redshift
Getting started with Amazon RedshiftGetting started with Amazon Redshift
Getting started with Amazon RedshiftAmazon Web Services
 
Percona Live 2014 - Scaling MySQL in AWS
Percona Live 2014 - Scaling MySQL in AWSPercona Live 2014 - Scaling MySQL in AWS
Percona Live 2014 - Scaling MySQL in AWSPythian
 
Altoros using no sql databases for interactive_applications
Altoros using no sql databases for interactive_applicationsAltoros using no sql databases for interactive_applications
Altoros using no sql databases for interactive_applicationsJeff Harris
 
Introduction to ClustrixDB
Introduction to ClustrixDBIntroduction to ClustrixDB
Introduction to ClustrixDBI Goo Lee
 
NO SQL: What, Why, How
NO SQL: What, Why, HowNO SQL: What, Why, How
NO SQL: What, Why, HowIgor Moochnick
 
عصر کلان داده، چرا و چگونه؟
عصر کلان داده، چرا و چگونه؟عصر کلان داده، چرا و چگونه؟
عصر کلان داده، چرا و چگونه؟datastack
 
Tech Talk Series, Part 2: Why is sharding not smart to do in MySQL?
Tech Talk Series, Part 2: Why is sharding not smart to do in MySQL?Tech Talk Series, Part 2: Why is sharding not smart to do in MySQL?
Tech Talk Series, Part 2: Why is sharding not smart to do in MySQL?Clustrix
 
Front Range PHP NoSQL Databases
Front Range PHP NoSQL DatabasesFront Range PHP NoSQL Databases
Front Range PHP NoSQL DatabasesJon Meredith
 
Benchmarking Couchbase Server for Interactive Applications
Benchmarking Couchbase Server for Interactive ApplicationsBenchmarking Couchbase Server for Interactive Applications
Benchmarking Couchbase Server for Interactive ApplicationsAltoros
 
Scaling SQL and NoSQL Databases in the Cloud
Scaling SQL and NoSQL Databases in the Cloud Scaling SQL and NoSQL Databases in the Cloud
Scaling SQL and NoSQL Databases in the Cloud RightScale
 
05 No SQL Sudarshan.ppt
05 No SQL Sudarshan.ppt05 No SQL Sudarshan.ppt
05 No SQL Sudarshan.pptAnandKonj1
 
No SQL Databases sdfghjkl;sdfghjkl;sdfghjkl;'
No SQL Databases sdfghjkl;sdfghjkl;sdfghjkl;'No SQL Databases sdfghjkl;sdfghjkl;sdfghjkl;'
No SQL Databases sdfghjkl;sdfghjkl;sdfghjkl;'sankarapu posibabu
 

Similar to Beyond Aurora. Scale-out SQL databases for AWS (20)

Scaling RDBMS on AWS- ClustrixDB @AWS Meetup 20160711
Scaling RDBMS on AWS- ClustrixDB @AWS Meetup 20160711Scaling RDBMS on AWS- ClustrixDB @AWS Meetup 20160711
Scaling RDBMS on AWS- ClustrixDB @AWS Meetup 20160711
 
NOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQLNOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQL
 
NOSQL in big data is the not only structure langua.pdf
NOSQL in big data is the not only structure langua.pdfNOSQL in big data is the not only structure langua.pdf
NOSQL in big data is the not only structure langua.pdf
 
NoSQL Options Compared
NoSQL Options ComparedNoSQL Options Compared
NoSQL Options Compared
 
Getting started with Amazon Redshift
Getting started with Amazon RedshiftGetting started with Amazon Redshift
Getting started with Amazon Redshift
 
No sq lv2
No sq lv2No sq lv2
No sq lv2
 
Percona Live 2014 - Scaling MySQL in AWS
Percona Live 2014 - Scaling MySQL in AWSPercona Live 2014 - Scaling MySQL in AWS
Percona Live 2014 - Scaling MySQL in AWS
 
Altoros using no sql databases for interactive_applications
Altoros using no sql databases for interactive_applicationsAltoros using no sql databases for interactive_applications
Altoros using no sql databases for interactive_applications
 
Introduction to ClustrixDB
Introduction to ClustrixDBIntroduction to ClustrixDB
Introduction to ClustrixDB
 
NO SQL: What, Why, How
NO SQL: What, Why, HowNO SQL: What, Why, How
NO SQL: What, Why, How
 
Deep Dive on Amazon Aurora
Deep Dive on Amazon AuroraDeep Dive on Amazon Aurora
Deep Dive on Amazon Aurora
 
عصر کلان داده، چرا و چگونه؟
عصر کلان داده، چرا و چگونه؟عصر کلان داده، چرا و چگونه؟
عصر کلان داده، چرا و چگونه؟
 
Tech Talk Series, Part 2: Why is sharding not smart to do in MySQL?
Tech Talk Series, Part 2: Why is sharding not smart to do in MySQL?Tech Talk Series, Part 2: Why is sharding not smart to do in MySQL?
Tech Talk Series, Part 2: Why is sharding not smart to do in MySQL?
 
Nosql seminar
Nosql seminarNosql seminar
Nosql seminar
 
Front Range PHP NoSQL Databases
Front Range PHP NoSQL DatabasesFront Range PHP NoSQL Databases
Front Range PHP NoSQL Databases
 
No sql
No sqlNo sql
No sql
 
Benchmarking Couchbase Server for Interactive Applications
Benchmarking Couchbase Server for Interactive ApplicationsBenchmarking Couchbase Server for Interactive Applications
Benchmarking Couchbase Server for Interactive Applications
 
Scaling SQL and NoSQL Databases in the Cloud
Scaling SQL and NoSQL Databases in the Cloud Scaling SQL and NoSQL Databases in the Cloud
Scaling SQL and NoSQL Databases in the Cloud
 
05 No SQL Sudarshan.ppt
05 No SQL Sudarshan.ppt05 No SQL Sudarshan.ppt
05 No SQL Sudarshan.ppt
 
No SQL Databases sdfghjkl;sdfghjkl;sdfghjkl;'
No SQL Databases sdfghjkl;sdfghjkl;sdfghjkl;'No SQL Databases sdfghjkl;sdfghjkl;sdfghjkl;'
No SQL Databases sdfghjkl;sdfghjkl;sdfghjkl;'
 

More from Clustrix

Tech Talk Series, Part 4: How do you achieve high availability in a MySQL env...
Tech Talk Series, Part 4: How do you achieve high availability in a MySQL env...Tech Talk Series, Part 4: How do you achieve high availability in a MySQL env...
Tech Talk Series, Part 4: How do you achieve high availability in a MySQL env...Clustrix
 
Tech Talk Series, Part 3: Why is your CFO right to demand you scale down MySQL?
Tech Talk Series, Part 3: Why is your CFO right to demand you scale down MySQL?Tech Talk Series, Part 3: Why is your CFO right to demand you scale down MySQL?
Tech Talk Series, Part 3: Why is your CFO right to demand you scale down MySQL?Clustrix
 
Demystifying Benchmarks: How to Use Them To Better Evaluate Databases
Demystifying Benchmarks: How to Use Them To Better Evaluate DatabasesDemystifying Benchmarks: How to Use Them To Better Evaluate Databases
Demystifying Benchmarks: How to Use Them To Better Evaluate DatabasesClustrix
 
Moving an E-commerce Site to AWS. A Case Study
Moving an  E-commerce Site to AWS. A Case StudyMoving an  E-commerce Site to AWS. A Case Study
Moving an E-commerce Site to AWS. A Case StudyClustrix
 
Scaling Techniques to Increase Magento Capacity
Scaling Techniques to Increase Magento CapacityScaling Techniques to Increase Magento Capacity
Scaling Techniques to Increase Magento CapacityClustrix
 
Supersizing Magento
Supersizing MagentoSupersizing Magento
Supersizing MagentoClustrix
 
Why Traditional Databases Fail so Miserably to Scale with E-Commerce Site Growth
Why Traditional Databases Fail so Miserably to Scale with E-Commerce Site GrowthWhy Traditional Databases Fail so Miserably to Scale with E-Commerce Site Growth
Why Traditional Databases Fail so Miserably to Scale with E-Commerce Site GrowthClustrix
 
E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.
E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.
E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.Clustrix
 
Clustrix Database Overview
Clustrix Database OverviewClustrix Database Overview
Clustrix Database OverviewClustrix
 
Clustrix Database Percona Ruby on Rails benchmark
Clustrix Database Percona Ruby on Rails benchmarkClustrix Database Percona Ruby on Rails benchmark
Clustrix Database Percona Ruby on Rails benchmarkClustrix
 

More from Clustrix (10)

Tech Talk Series, Part 4: How do you achieve high availability in a MySQL env...
Tech Talk Series, Part 4: How do you achieve high availability in a MySQL env...Tech Talk Series, Part 4: How do you achieve high availability in a MySQL env...
Tech Talk Series, Part 4: How do you achieve high availability in a MySQL env...
 
Tech Talk Series, Part 3: Why is your CFO right to demand you scale down MySQL?
Tech Talk Series, Part 3: Why is your CFO right to demand you scale down MySQL?Tech Talk Series, Part 3: Why is your CFO right to demand you scale down MySQL?
Tech Talk Series, Part 3: Why is your CFO right to demand you scale down MySQL?
 
Demystifying Benchmarks: How to Use Them To Better Evaluate Databases
Demystifying Benchmarks: How to Use Them To Better Evaluate DatabasesDemystifying Benchmarks: How to Use Them To Better Evaluate Databases
Demystifying Benchmarks: How to Use Them To Better Evaluate Databases
 
Moving an E-commerce Site to AWS. A Case Study
Moving an  E-commerce Site to AWS. A Case StudyMoving an  E-commerce Site to AWS. A Case Study
Moving an E-commerce Site to AWS. A Case Study
 
Scaling Techniques to Increase Magento Capacity
Scaling Techniques to Increase Magento CapacityScaling Techniques to Increase Magento Capacity
Scaling Techniques to Increase Magento Capacity
 
Supersizing Magento
Supersizing MagentoSupersizing Magento
Supersizing Magento
 
Why Traditional Databases Fail so Miserably to Scale with E-Commerce Site Growth
Why Traditional Databases Fail so Miserably to Scale with E-Commerce Site GrowthWhy Traditional Databases Fail so Miserably to Scale with E-Commerce Site Growth
Why Traditional Databases Fail so Miserably to Scale with E-Commerce Site Growth
 
E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.
E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.
E-Commerce Success is a Balancing Act. Ensure Success with ClustrixDB.
 
Clustrix Database Overview
Clustrix Database OverviewClustrix Database Overview
Clustrix Database Overview
 
Clustrix Database Percona Ruby on Rails benchmark
Clustrix Database Percona Ruby on Rails benchmarkClustrix Database Percona Ruby on Rails benchmark
Clustrix Database Percona Ruby on Rails benchmark
 

Recently uploaded

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 

Recently uploaded (20)

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 

Beyond Aurora. Scale-out SQL databases for AWS

  • 1. © 2015 CLUSTRIX The First Scale-out SQL Database Engineered for Today’s Cloud Beyond Aurora. Scale-out SQL databases for AWS
  • 2. Magnus Data SCALE OUT RDBMS 3/4/2016 MagnusData
  • 3. Agenda  Market Landscape of DB market  Options to Scale DB  Scale-Out Architecture  Comparisons of solutions for high transaction relational databases 3/4/2016
  • 4. Generalized and Specialized 3/4/2016 High Concurrency/Write heavy /Real Time Analytics Historical Analytics Exploratory Transactional Analytics Traditional Databases No SQL DW/Analytical DBMS Operational System/OLTP (New SQL) Hadoop
  • 5. Scale-Up vs. Scale-Out 3/4/2016 Scale-Out databases Transactions Per Second LatencyHigh High Scale-Up Databases (like Aurora and MySQL)
  • 6. RDBMS Scaling Techniques  Scale-Up  Master Slave  Master Master  MySQL Clustering Technologies  Sharding  Scale-Out 3/4/2016
  • 7. Options to Scale DBMS 3/4/2016 DBMS Scale Out e.g., MongoDB No transactions May have weak consistency (CAP) Application involves DB Coding e.g. ClustrixDB ACID Proven Scalability (Reads and Writes) Shared Nothing Scale Up e.g., Aurora Reads Scale limited scalability on writes Not Shared nothing scale out
  • 8. Scaling-Up  Keep increasing the size of the (single) database server  Pros  Simple, no application changes needed  Cons  Expensive. At some point, you’re paying 5x for 2x the performance  ‘Exotic’ hardware (128 cores and above) become price prohibitive  Eventually you ‘hit the wall’, and you literally cannot scale-up anymore 8
  • 9. Scaling Reads: Master/Slave  Add a ‘Slave’ read-server(s) to your ‘Master’ database server  Pros  Reasonably simple to implement.  Read/write fan-out can be done at the proxy level  Cons  Only adds Read performance  Data consistency issues can occur, especially if the application isn’t coded to ensure reads from the slave are consistent with reads from the master 9
  • 10. Scaling Writes: Master/Master 10  Add additional ‘Master’(s) to your ‘Master’ database server  Pros  Adds Write scaling without needing to shard  Cons  Adds write scaling at the cost of read-slaves  Adding read-slaves would add even more latency  Application changes are required to ensure data consistency / conflict resolution
  • 11. Scaling Reads & Writes: Sharding 11 SHARDO1 SHARDO2 SHARDO3 SHARDO4  Partitioning tables across separate database servers  Pros  Adds both write and read scaling  Cons  Loses the ability of an RDBMS to manage transactionality, referential integrity and ACID  ACID compliance & transactionality must be managed at the application level  Consistent backups across all the shards are very hard to manage  Read and Writes can be skewed / unbalanced  Application changes can be significant A - K L - O P - S T - Z
  • 12. Scaling Reads & Writes: MySQL Cluster  Provides shared-nothing clustering and auto-sharding for MySQL. (designed for Telco deployments: minimal cross-node transactions, HA emphasis)  Pros  Distributed, multi-master model  Provides high availability and high throughput  Cons  Only supports read-committed isolation  Long-running transactions can block a node restart  SBR replication not supported  Range scans are expensive and lower performance than MySQL  Unclear how it scales with many nodes 12
  • 13. Application Workload Partitioning 13  Partition entire application + RDBMS stack across several “pods”  Pros  Adds both write and read scaling  Flexible: can keep scaling with addition of pods  Cons  No data consistency across pods (only suited for cases where it is not needed)  High overhead in DBMS maintenance and upgrade  Queries / Reports across all pods can be very complex  Complex environment to setup and support APP APP APP APP APP APP
  • 14. DBMS Capacity, Elasticity and Resiliency 14 Scale-up Master – Slave Master – Master MySQL Cluster Sharding Scale-Out DBMS Scaling Many cores – very expensive Reads Only Read / Write Read / Write Unbalanced Read/Writes Read / Write Capacity Single Point Failure Fail-over Yes Yes Multiple points of failure Yes ResiliencyElasticity No No No No No Yes None Yes – for read scale High – update conflict None (or minor) Very High None Application Impact
  • 15. DBMS Architecture-Scale out 3/4/2016 Shared Nothing Architecture Compiler Map Engine Data Compiler Map Engine Data Compiler Map Engine Data Each Node Contains:  Query Parser/Planner: distribute partial query fragments to the nodes.  Data Map: all nodes metadata about data across the cluster  Database Engine: all nodes can perform all database operations (no leader, aggregator, leaf, data-only, etc nodes)  Data: Table Distributed: All table auto- redistributed
  • 16. BillionsofRows Database Tables S1 S2 S2 S3 S3 S4 S4 S5 S5 Intelligent Data Distribution 16 S1 ClustrixDB  Tables Auto Distributed across nodes  Tunable amount of redundancy of data across nodes  Tables are auto distributed, auto-protected
  • 17. Query Distributed Query Processing 17 ClustrixDB Load Balancer TRXTRXTRX  Queries are fielded by any peer node  Routed to node holding the data  Complex queries are split into steps and processed in parallel  Automatically distributed for optimized performance  All nodes handle writes and reads  Result is aggregated and returned to the user
  • 18. DBMS Capacity, Elasticity and Resiliency 18 Features ClustrixDB Aurora Write Scalability Writes scales by adding nodes Cannot add write nodes High Concurrency Latency Low with High concurrency Latency climbs quickly with high concurrency ACID Yes Yes On-Demand Write Scale Yes No Automatically Distributed queries Yes: No Application changes No: Read/Write fanout needed. Write contention on Master Cloud/On Premises Yes No, only AWS Cloud Shared Nothing Storage Yes: Parallel data access No: Contention at high write concurrency
  • 19. Benchmark Results 3/4/2016 0 10 20 30 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000 AverageLatency(ms) Throughput (tps) Sysbench OLTP 90:10 Mix 0 10 20 30 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000 AverageLatency(ms) Throughput (tps) Sysbench OLTP 90:10 Mix Clustrix 4 Node Aurora Mysql RDS
  • 20. Scalability Test 3/4/2016 0 10 20 30 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000 AverageLatency(ms) Throughput (tps) Sysbench OLTP 90:10 Mix

Editor's Notes

  1. https://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-replication-issues.html https://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-limitations-transactions.html https://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-limitations-unsupported.html
  2. Simple queries Fielded by any node Routed to data node Complex queries Split into query fragments Process fragments in parallel
  3. Building a scalable distributed database requires two things Distributing the data intelligently Moving the queries to the data