SlideShare a Scribd company logo
1 of 26
Download to read offline
Ariff Kassam | Chief Technology Officer
3 Things You Need to Know When
Assessing Database Scalability
November 12th, 2019
Copyright © 2019 NuoDB, Inc. 2 |
Meet the Speaker
Ariff Kassam
Chief Technology Officer
NuoDB
Copyright © 2019 NuoDB, Inc.
+ Database Availability & Scalability
+ Scaling Options for Databases
+ Trade Offs
+ About NuoDB
+ Q&A
3 |
Agenda
Copyright © 2019 NuoDB, Inc.
+ Database availability, performance, and scalability have
always been challenging for organizations
+ Many have accepted the fact that it requires expensive
hardware and software to deliver a robust database
solution
+ However, recent innovations in database technologies have
expanded the options available to organizations
+ These innovations can provide better availability and
scalability at lower TCO
+ Need to understand the tradeoffs that need to be made
4 |
Holy Grail: Database Availability & Scalability
Copyright © 2019 NuoDB, Inc.
+ For stateless
processing, scale out
architectures are
standard practices
▪ Costs increase with
performance
+ For databases, scale up
architecture is the norm
▪ Costs increase faster
than performance
+ Kubernetes adds additional
complexity 5 |
Scale Up vs Scale Out
Source: Paris Technology
Copyright © 2019 NuoDB, Inc.
+ As data volumes and usage grows, the only approach to
address database performance issues is to throw more
hardware at the problem
+ Add more CPUs, memory, and/or disk to improve performance
+ Server upgrades require maintenance windows, so servers are
sized for peak performance requirements in the future
+ Hardware needs to be replicated for any HA or DR
environments
+ Results in significant costs
6 |
Scale Up
Copyright © 2019 NuoDB, Inc.
+ Newer database solutions provide scale out capabilities
▪ Dynamic provisioning
▪ Lower TCO
▪ Better availability
+ But databases are stateful systems
+ Scale out requires the ability to distribute the data and
workloads across multiple systems
+ To enable scale out, certain trade offs need to be made
7 |
Scale Out
Copyright © 2019 NuoDB, Inc.
+ NoSQL
+ Traditional SQL architectures
▪ Read replicas
▪ Database sharding
+ Storage focused SQL architectures
▪ Clustering / shared disk
▪ Distributed storage
+ Distributed SQL
▪ Automatic partitioning
▪ Durable distributed cache
8 |
Database Scale Out Options
Copyright © 2019 NuoDB, Inc.
+ Innovation driven by the need to address the
limitations with scalability and availability of
traditional SQL databases
+ CAP theorem
▪ For distributed systems, when a
Partition occurs, the system must
choose to save Consistency or
provide Availability
+ Different types of NoSQL models
depending on use cases
9 |
NoSQL
Copyright © 2019 NuoDB, Inc.
+ Works well for certain types of applications and use
cases
▪ Event logging, content management, metadata
management, social data, IoT, real-time analytics, etc.
+ But for existing applications written to use traditional
SQL databases needing ACID transactions
▪ High risk to convert to NoSQL
▪ Need to re-develop consistency and transactional
assumptions, data model, performance, and operational
management, etc.
10 |
NoSQL
Copyright © 2019 NuoDB, Inc.
+ Originally developed to provide faster
failover in case of failure of master
+ Database copies that are kept in sync
with the master database and used for
read only queries
+ Need a load balancer to auto
route traffic or segregate
workload
+ Does not address write scale out and
requires an outage on failover
11 |
Traditional SQL: Read Replicas
App
Reads &
Writes
BI
Reads
Only
Master Read
Replicas
Copyright © 2019 NuoDB, Inc.
+ Standard technique to improve performance for large SQL
databases
+ A single large database is split up into multiple smaller
independent databases
▪ All databases have the same schema
▪ Data is split across the smaller
databases
+ Improve performance
+ Application needs to be shardable
and harder to manage/operate
+ Does not improve availability
+ Hard to add shards dynamically
12 |
Traditional SQL: Database Sharding
Copyright © 2019 NuoDB, Inc.
+ Originally developed to remove the need for an outage
on node failure
+ This architecture supports multiple read/write database
nodes, all storing the data on the same shared disk
+ Works well for read scale out
+ Has limitations for write scale out
+ Also requires specialized hardware and
networking not available in the cloud
+ Examples: Oracle RAC, IBM PureScale
13 |
Storage: Clustered/Shared Storage
App App App
Copyright © 2019 NuoDB, Inc.
+ Improvement on the Read Replica model
+ Newer database storage architectures to improve I/O
performance and data availability
+ Used by AWS Aurora,
Azure SQL Hyperscale
14 |
Storage: Distributed Storage
Copyright © 2019 NuoDB, Inc.
+ Newer systems automatically partition all of the data
across multiple nodes
+ One node is the “master” for a data partition
+ All reads/writes are must be processed by the
master
+ Replicas of that data are distributed to
other nodes for availability
+ Need to be careful of master locality and
Workload
+ Adding new nodes may require re-partitioning
15 |
Distributed SQL: Auto Partitioning
Copyright © 2019 NuoDB, Inc. 16 |
Distributed SQL: Durable Distributed Cache
+ Modern cloud architecture with split query processing
(compute) and storage management
+ Each layer can be scaled out independently
+ Distributed cache does not require partitioning of the
workload
+ Data is automatically
replicated at the
storage layer
Database Process
Storage Management
Query Processing
Query Processing
Storage Management
Copyright © 2019 NuoDB, Inc.
+ All architectures that support scale out make certain
tradeoffs
+ Each have advantages and disadvantages
+ Need to review your use case and requirements to
determine which approach would be best for you
17 |
Scale Out Tradeoffs
Copyright © 2019 NuoDB, Inc. 18 |
Trade Off Summary
Traditional SQL Storage Focused Distributed SQL
Read
Replicas
Database
Sharding
Clustered Distributed
Storage
Auto
Partitioning
Durable
Distributed Cache
Read Scale Out Y Y Y Y Y Y
Write Scale Out N Y N N Y Y
Workload Independent Y N Y Y N Y
Low Latency Y N Y Y N Y
Availability N N Y Y Y Y
Commodity Hardware Y Y N N Y/N Y
Complexity Low High Medium Low Medium Medium
Copyright © 2019 NuoDB, Inc.
NuoDB
19 |
Modern cloud-native, cloud-agnostic
distributed SQL database for high
throughput transactional workloads
+ On-demand scale out and scale in
+ Continuous availability protection
against planned and unplanned
outages for HA & DR
+ Hybrid or multi-cloud deployments in
any public or private cloud and on
physical, virtual, and containerized
environments
+ ANSI SQL interface
Storage
Mgmt
Query
Processing
App App
Traditional
RDBMS
Architecture
NuoDB Architecture
Durability
Connection Mgmt
SQL Parser
SQL Optimizer
Transaction Mgmt
Transaction Engine (TE)
Storage Manager (SM)
TE
SM SM
App App
TE
Copyright © 2019 NuoDB 20 |
TE and/or SM Failure
1. Working Database
SMSM
TE TE TE
App App App
2. TE and SM failure
SMSM
TE TE TE
App App App
3. Application reconnects to surviving TE
SMSM
TE TE TE
App App App
Database Availability
Storage Manager (SM)SMTransaction Engine (TE)TEApplicationApp
SM SM
Copyright © 2019 NuoDB
Multi-Tier Scalability
Web/Mobile Application
+ Read-dominated
+ Cache-bound
+ High concurrency
+ Many simple queries
+ Solution: Scale-out Compute Tier
OLTP Application
+ Mixed read/insert/update/delete
+ Compute & IO bound
+ Medium/High concurrency
+ Stable working set
+ Solution: Scale-out Compute & Storage Tiers
TE TE TE TE TE
SM SM
TE TE TE TE TE
SM SMSM SM
21 |
Copyright © 2019 NuoDB
Multi-Tier Scalability
Logging Application
+ Insert dominated
+ IO bound
+ Large data size
+ Solution: Scale-out Storage Tier
HTAP Application
+ Mixed read/insert/update/delete
+ Compute & IO bound
+ Medium/High concurrency
+ Resource contention
+ Solution: Dedicated Analytical Nodes
TE TE TE TE TE
SM SM
TE TE
SM SMSM SM
22 |
Copyright © 2019 NuoDB, Inc.
Distributed
Architecture
Active-Active
Scale Out
Automated Ops
Dynamic
Caching
23 |
Benefits
Modern
architecture
separating
compute and
storage
Zero failover time
(RTO=0) for always
on protection
Automated
deployment and
operations using
Kubernetes
Operators
Address dynamic
performance
requirements with
on-demand scale
out and scale in
Optimize
performance/cost trade
offs with dynamic
memory caching
Copyright © 2019 NuoDB, Inc.
+ Various options available to address database
scalability limitations of the past
+ Each has different trade offs with different advantages
or disadvantages
+ Match solutions to your requirements, use cases, and
workloads
+ NuoDB is a Distributed SQL solution for existing low
latency enterprise critical applications moving to the
cloud
24 |
Summary
Copyright © 2019 NuoDB, Inc.
Next Steps
Useful resources:
+ The Pros and Cons of Database Scaling
Options
+ The Rising Need for Database Scalability
+ What do we mean by Database Scalability?
+ What are the Database Scalability
methods?
+ What is the Elastic SQL approach?
25 |
Questions?

More Related Content

More from NuoDB

Introduction to NuoDB - March 2018
Introduction to NuoDB - March 2018Introduction to NuoDB - March 2018
Introduction to NuoDB - March 2018NuoDB
 
Transforming Retail Banking: Competitive Advantage through Microservices
Transforming Retail Banking: Competitive Advantage through MicroservicesTransforming Retail Banking: Competitive Advantage through Microservices
Transforming Retail Banking: Competitive Advantage through MicroservicesNuoDB
 
451 Research + NuoDB: What It Means to be a Container-Native SQL Database
451 Research + NuoDB: What It Means to be a Container-Native SQL Database451 Research + NuoDB: What It Means to be a Container-Native SQL Database
451 Research + NuoDB: What It Means to be a Container-Native SQL DatabaseNuoDB
 
Microservices Applications: Challenges and Best Practices When Deploying SQL-...
Microservices Applications: Challenges and Best Practices When Deploying SQL-...Microservices Applications: Challenges and Best Practices When Deploying SQL-...
Microservices Applications: Challenges and Best Practices When Deploying SQL-...NuoDB
 
Building Cloud-Native Applications with a Container-Native SQL Database in th...
Building Cloud-Native Applications with a Container-Native SQL Database in th...Building Cloud-Native Applications with a Container-Native SQL Database in th...
Building Cloud-Native Applications with a Container-Native SQL Database in th...NuoDB
 
5 Steps for Migrating Relational Databases to Next-Gen Architectures
5 Steps for Migrating Relational Databases to Next-Gen Architectures5 Steps for Migrating Relational Databases to Next-Gen Architectures
5 Steps for Migrating Relational Databases to Next-Gen ArchitecturesNuoDB
 
NuoDB 3.0: Getting Started with Community Edition
NuoDB 3.0: Getting Started with Community EditionNuoDB 3.0: Getting Started with Community Edition
NuoDB 3.0: Getting Started with Community EditionNuoDB
 
Cloud Database Migration Made Easy: Migrating MySQL to NuoDB
Cloud Database Migration Made Easy: Migrating MySQL to NuoDBCloud Database Migration Made Easy: Migrating MySQL to NuoDB
Cloud Database Migration Made Easy: Migrating MySQL to NuoDBNuoDB
 
Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)
Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)
Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)NuoDB
 
Reasons to Deploy an Elastic SQL Database
Reasons to Deploy an Elastic SQL DatabaseReasons to Deploy an Elastic SQL Database
Reasons to Deploy an Elastic SQL DatabaseNuoDB
 
Getting Started with NuoDB Community Edition
Getting Started with NuoDB Community Edition Getting Started with NuoDB Community Edition
Getting Started with NuoDB Community Edition NuoDB
 
Key Database Criteria for Cloud Applications
Key Database Criteria for Cloud ApplicationsKey Database Criteria for Cloud Applications
Key Database Criteria for Cloud ApplicationsNuoDB
 
New york-breakfast-seminar
New york-breakfast-seminarNew york-breakfast-seminar
New york-breakfast-seminarNuoDB
 
Choosing The Right Database For Your Cloud Application
Choosing The Right Database For Your Cloud ApplicationChoosing The Right Database For Your Cloud Application
Choosing The Right Database For Your Cloud ApplicationNuoDB
 
California Breakfast Seminar
California Breakfast SeminarCalifornia Breakfast Seminar
California Breakfast SeminarNuoDB
 
London Breakfast Seminar
London Breakfast SeminarLondon Breakfast Seminar
London Breakfast SeminarNuoDB
 
LTE: Building next-gen application services for mobile telecoms
LTE: Building next-gen application services for mobile telecomsLTE: Building next-gen application services for mobile telecoms
LTE: Building next-gen application services for mobile telecomsNuoDB
 
Dallas Breakfast Seminar
Dallas Breakfast SeminarDallas Breakfast Seminar
Dallas Breakfast SeminarNuoDB
 
Cambridge Breakfast Seminar
Cambridge Breakfast SeminarCambridge Breakfast Seminar
Cambridge Breakfast SeminarNuoDB
 
Future of Cloud: Insights From the Front Line
Future of Cloud: Insights From the Front LineFuture of Cloud: Insights From the Front Line
Future of Cloud: Insights From the Front LineNuoDB
 

More from NuoDB (20)

Introduction to NuoDB - March 2018
Introduction to NuoDB - March 2018Introduction to NuoDB - March 2018
Introduction to NuoDB - March 2018
 
Transforming Retail Banking: Competitive Advantage through Microservices
Transforming Retail Banking: Competitive Advantage through MicroservicesTransforming Retail Banking: Competitive Advantage through Microservices
Transforming Retail Banking: Competitive Advantage through Microservices
 
451 Research + NuoDB: What It Means to be a Container-Native SQL Database
451 Research + NuoDB: What It Means to be a Container-Native SQL Database451 Research + NuoDB: What It Means to be a Container-Native SQL Database
451 Research + NuoDB: What It Means to be a Container-Native SQL Database
 
Microservices Applications: Challenges and Best Practices When Deploying SQL-...
Microservices Applications: Challenges and Best Practices When Deploying SQL-...Microservices Applications: Challenges and Best Practices When Deploying SQL-...
Microservices Applications: Challenges and Best Practices When Deploying SQL-...
 
Building Cloud-Native Applications with a Container-Native SQL Database in th...
Building Cloud-Native Applications with a Container-Native SQL Database in th...Building Cloud-Native Applications with a Container-Native SQL Database in th...
Building Cloud-Native Applications with a Container-Native SQL Database in th...
 
5 Steps for Migrating Relational Databases to Next-Gen Architectures
5 Steps for Migrating Relational Databases to Next-Gen Architectures5 Steps for Migrating Relational Databases to Next-Gen Architectures
5 Steps for Migrating Relational Databases to Next-Gen Architectures
 
NuoDB 3.0: Getting Started with Community Edition
NuoDB 3.0: Getting Started with Community EditionNuoDB 3.0: Getting Started with Community Edition
NuoDB 3.0: Getting Started with Community Edition
 
Cloud Database Migration Made Easy: Migrating MySQL to NuoDB
Cloud Database Migration Made Easy: Migrating MySQL to NuoDBCloud Database Migration Made Easy: Migrating MySQL to NuoDB
Cloud Database Migration Made Easy: Migrating MySQL to NuoDB
 
Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)
Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)
Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)
 
Reasons to Deploy an Elastic SQL Database
Reasons to Deploy an Elastic SQL DatabaseReasons to Deploy an Elastic SQL Database
Reasons to Deploy an Elastic SQL Database
 
Getting Started with NuoDB Community Edition
Getting Started with NuoDB Community Edition Getting Started with NuoDB Community Edition
Getting Started with NuoDB Community Edition
 
Key Database Criteria for Cloud Applications
Key Database Criteria for Cloud ApplicationsKey Database Criteria for Cloud Applications
Key Database Criteria for Cloud Applications
 
New york-breakfast-seminar
New york-breakfast-seminarNew york-breakfast-seminar
New york-breakfast-seminar
 
Choosing The Right Database For Your Cloud Application
Choosing The Right Database For Your Cloud ApplicationChoosing The Right Database For Your Cloud Application
Choosing The Right Database For Your Cloud Application
 
California Breakfast Seminar
California Breakfast SeminarCalifornia Breakfast Seminar
California Breakfast Seminar
 
London Breakfast Seminar
London Breakfast SeminarLondon Breakfast Seminar
London Breakfast Seminar
 
LTE: Building next-gen application services for mobile telecoms
LTE: Building next-gen application services for mobile telecomsLTE: Building next-gen application services for mobile telecoms
LTE: Building next-gen application services for mobile telecoms
 
Dallas Breakfast Seminar
Dallas Breakfast SeminarDallas Breakfast Seminar
Dallas Breakfast Seminar
 
Cambridge Breakfast Seminar
Cambridge Breakfast SeminarCambridge Breakfast Seminar
Cambridge Breakfast Seminar
 
Future of Cloud: Insights From the Front Line
Future of Cloud: Insights From the Front LineFuture of Cloud: Insights From the Front Line
Future of Cloud: Insights From the Front Line
 

Recently uploaded

Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Scott Andery
 

Recently uploaded (20)

Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
 

3 Things You Need to Know When Assessing Database Scalability

  • 1. Ariff Kassam | Chief Technology Officer 3 Things You Need to Know When Assessing Database Scalability November 12th, 2019
  • 2. Copyright © 2019 NuoDB, Inc. 2 | Meet the Speaker Ariff Kassam Chief Technology Officer NuoDB
  • 3. Copyright © 2019 NuoDB, Inc. + Database Availability & Scalability + Scaling Options for Databases + Trade Offs + About NuoDB + Q&A 3 | Agenda
  • 4. Copyright © 2019 NuoDB, Inc. + Database availability, performance, and scalability have always been challenging for organizations + Many have accepted the fact that it requires expensive hardware and software to deliver a robust database solution + However, recent innovations in database technologies have expanded the options available to organizations + These innovations can provide better availability and scalability at lower TCO + Need to understand the tradeoffs that need to be made 4 | Holy Grail: Database Availability & Scalability
  • 5. Copyright © 2019 NuoDB, Inc. + For stateless processing, scale out architectures are standard practices ▪ Costs increase with performance + For databases, scale up architecture is the norm ▪ Costs increase faster than performance + Kubernetes adds additional complexity 5 | Scale Up vs Scale Out Source: Paris Technology
  • 6. Copyright © 2019 NuoDB, Inc. + As data volumes and usage grows, the only approach to address database performance issues is to throw more hardware at the problem + Add more CPUs, memory, and/or disk to improve performance + Server upgrades require maintenance windows, so servers are sized for peak performance requirements in the future + Hardware needs to be replicated for any HA or DR environments + Results in significant costs 6 | Scale Up
  • 7. Copyright © 2019 NuoDB, Inc. + Newer database solutions provide scale out capabilities ▪ Dynamic provisioning ▪ Lower TCO ▪ Better availability + But databases are stateful systems + Scale out requires the ability to distribute the data and workloads across multiple systems + To enable scale out, certain trade offs need to be made 7 | Scale Out
  • 8. Copyright © 2019 NuoDB, Inc. + NoSQL + Traditional SQL architectures ▪ Read replicas ▪ Database sharding + Storage focused SQL architectures ▪ Clustering / shared disk ▪ Distributed storage + Distributed SQL ▪ Automatic partitioning ▪ Durable distributed cache 8 | Database Scale Out Options
  • 9. Copyright © 2019 NuoDB, Inc. + Innovation driven by the need to address the limitations with scalability and availability of traditional SQL databases + CAP theorem ▪ For distributed systems, when a Partition occurs, the system must choose to save Consistency or provide Availability + Different types of NoSQL models depending on use cases 9 | NoSQL
  • 10. Copyright © 2019 NuoDB, Inc. + Works well for certain types of applications and use cases ▪ Event logging, content management, metadata management, social data, IoT, real-time analytics, etc. + But for existing applications written to use traditional SQL databases needing ACID transactions ▪ High risk to convert to NoSQL ▪ Need to re-develop consistency and transactional assumptions, data model, performance, and operational management, etc. 10 | NoSQL
  • 11. Copyright © 2019 NuoDB, Inc. + Originally developed to provide faster failover in case of failure of master + Database copies that are kept in sync with the master database and used for read only queries + Need a load balancer to auto route traffic or segregate workload + Does not address write scale out and requires an outage on failover 11 | Traditional SQL: Read Replicas App Reads & Writes BI Reads Only Master Read Replicas
  • 12. Copyright © 2019 NuoDB, Inc. + Standard technique to improve performance for large SQL databases + A single large database is split up into multiple smaller independent databases ▪ All databases have the same schema ▪ Data is split across the smaller databases + Improve performance + Application needs to be shardable and harder to manage/operate + Does not improve availability + Hard to add shards dynamically 12 | Traditional SQL: Database Sharding
  • 13. Copyright © 2019 NuoDB, Inc. + Originally developed to remove the need for an outage on node failure + This architecture supports multiple read/write database nodes, all storing the data on the same shared disk + Works well for read scale out + Has limitations for write scale out + Also requires specialized hardware and networking not available in the cloud + Examples: Oracle RAC, IBM PureScale 13 | Storage: Clustered/Shared Storage App App App
  • 14. Copyright © 2019 NuoDB, Inc. + Improvement on the Read Replica model + Newer database storage architectures to improve I/O performance and data availability + Used by AWS Aurora, Azure SQL Hyperscale 14 | Storage: Distributed Storage
  • 15. Copyright © 2019 NuoDB, Inc. + Newer systems automatically partition all of the data across multiple nodes + One node is the “master” for a data partition + All reads/writes are must be processed by the master + Replicas of that data are distributed to other nodes for availability + Need to be careful of master locality and Workload + Adding new nodes may require re-partitioning 15 | Distributed SQL: Auto Partitioning
  • 16. Copyright © 2019 NuoDB, Inc. 16 | Distributed SQL: Durable Distributed Cache + Modern cloud architecture with split query processing (compute) and storage management + Each layer can be scaled out independently + Distributed cache does not require partitioning of the workload + Data is automatically replicated at the storage layer Database Process Storage Management Query Processing Query Processing Storage Management
  • 17. Copyright © 2019 NuoDB, Inc. + All architectures that support scale out make certain tradeoffs + Each have advantages and disadvantages + Need to review your use case and requirements to determine which approach would be best for you 17 | Scale Out Tradeoffs
  • 18. Copyright © 2019 NuoDB, Inc. 18 | Trade Off Summary Traditional SQL Storage Focused Distributed SQL Read Replicas Database Sharding Clustered Distributed Storage Auto Partitioning Durable Distributed Cache Read Scale Out Y Y Y Y Y Y Write Scale Out N Y N N Y Y Workload Independent Y N Y Y N Y Low Latency Y N Y Y N Y Availability N N Y Y Y Y Commodity Hardware Y Y N N Y/N Y Complexity Low High Medium Low Medium Medium
  • 19. Copyright © 2019 NuoDB, Inc. NuoDB 19 | Modern cloud-native, cloud-agnostic distributed SQL database for high throughput transactional workloads + On-demand scale out and scale in + Continuous availability protection against planned and unplanned outages for HA & DR + Hybrid or multi-cloud deployments in any public or private cloud and on physical, virtual, and containerized environments + ANSI SQL interface Storage Mgmt Query Processing App App Traditional RDBMS Architecture NuoDB Architecture Durability Connection Mgmt SQL Parser SQL Optimizer Transaction Mgmt Transaction Engine (TE) Storage Manager (SM) TE SM SM App App TE
  • 20. Copyright © 2019 NuoDB 20 | TE and/or SM Failure 1. Working Database SMSM TE TE TE App App App 2. TE and SM failure SMSM TE TE TE App App App 3. Application reconnects to surviving TE SMSM TE TE TE App App App Database Availability Storage Manager (SM)SMTransaction Engine (TE)TEApplicationApp SM SM
  • 21. Copyright © 2019 NuoDB Multi-Tier Scalability Web/Mobile Application + Read-dominated + Cache-bound + High concurrency + Many simple queries + Solution: Scale-out Compute Tier OLTP Application + Mixed read/insert/update/delete + Compute & IO bound + Medium/High concurrency + Stable working set + Solution: Scale-out Compute & Storage Tiers TE TE TE TE TE SM SM TE TE TE TE TE SM SMSM SM 21 |
  • 22. Copyright © 2019 NuoDB Multi-Tier Scalability Logging Application + Insert dominated + IO bound + Large data size + Solution: Scale-out Storage Tier HTAP Application + Mixed read/insert/update/delete + Compute & IO bound + Medium/High concurrency + Resource contention + Solution: Dedicated Analytical Nodes TE TE TE TE TE SM SM TE TE SM SMSM SM 22 |
  • 23. Copyright © 2019 NuoDB, Inc. Distributed Architecture Active-Active Scale Out Automated Ops Dynamic Caching 23 | Benefits Modern architecture separating compute and storage Zero failover time (RTO=0) for always on protection Automated deployment and operations using Kubernetes Operators Address dynamic performance requirements with on-demand scale out and scale in Optimize performance/cost trade offs with dynamic memory caching
  • 24. Copyright © 2019 NuoDB, Inc. + Various options available to address database scalability limitations of the past + Each has different trade offs with different advantages or disadvantages + Match solutions to your requirements, use cases, and workloads + NuoDB is a Distributed SQL solution for existing low latency enterprise critical applications moving to the cloud 24 | Summary
  • 25. Copyright © 2019 NuoDB, Inc. Next Steps Useful resources: + The Pros and Cons of Database Scaling Options + The Rising Need for Database Scalability + What do we mean by Database Scalability? + What are the Database Scalability methods? + What is the Elastic SQL approach? 25 |