Ariff Kassam, VP of Products – akassam@nuodb.com
Christina Wong, Dir. of Product Marketing – cwong@nuodb.com
ELASTIC SQL: FROM OXYMORON
TO EMERGING REALITY
June 23, 2017
Copyright © 2017 NuoDB
ELASTIC SQL DATABASE
+ Traditional SQL databases are not suited for modern
system architectures or development practices
 Doesn’t fit with scale out, container, dynamic orchestration
architectures
+ NoSQL databases better suited for new applications but
hard to migrate existing applications
+ Market need: Provide scale out, containerization, HA for
1
Copyright © 2017 NuoDB
ARCHITECTURE EVOLUTION
Mainframe
Presentation
App Logic
Data Mgmt
Monolithic
Open Systems
Presentation
App Logic
Data Mgmt
Client / Server
Presentation
App Logic
Data Mgmt
N-Tier
Internet Cloud
Microservices
Data Mgmt
Infrastructure
Architectures
Application
Architectures
2
Copyright © 2017 NuoDB
DATABASE ARCHITECTURES
+ Relational database architectures have not changed
significantly with infrastructure advances
 Hardware: Scale up or shared disk
 Application: 2 Phase Commit or Sharding
3
Scale Up Shared Disk 2 Phase Commit Sharded Database
Copyright © 2017 NuoDB
WHAT ABOUT NOSQL?
+ Provides scale out with automatic sharding
+ Can be an option for new applications or existing
applications with significant
re-write
+ Challenging trade-offs
 Account for eventual consistency
 Code for transactional semantics
 Code for certain queries (joins, etc) with static logic
 Manage and maintain data structures
4
Copyright © 2017 NuoDB
TODAY’S DATABASE MARKET PROBLEM
Elastic SQL
Elastic SQL: The benefits of
Traditional Databases & The Cloud
SQL
Logic
ACID:
data
consistency
Zero
Downtime
Elastic
scale-out
SQL database
abstraction
Virtualization,
Commodity
& Cloud
Traditional RDBMS
NoSQL
MARKET GAP
SQL
Logic
ACID:
data
consistency
SQL database
abstraction
Zero
Downtime
Elastic
scale-out
Virtualization,
Commodity
& Cloud
5
Copyright © 2017 NuoDB
ELASTIC SQL DATABASE
An elastic SQL database
combines the scale-out simplicity, elasticity, and
continuous availability that cloud applications require with
the transactional consistency and durability that
databases of record demand.
7
Copyright © 2017 NuoDB
THE EMERGING ELASTIC SQL MARKET
Traditional
RDBMS
NoSQL
Elastic SQL
Business Database of
Record ✓ ✗ ✓ ✓ ✓
ANSI-Standard SQL ✓ ✗ (✓) (✓) ✓
Migrate existing SQL apps ✓ ✗ ✗ ✗ ✓
In-memory performance ✓ ✓ ✓ ✗ ✓
Simple elastic scale-out/in
(reads and writes) ✗ ✓ ✓ ✓ ✓
Zero downtime ✗ ✓ ✓ ✓ ✓
Deployment flexibility
(on-prem, private or public cloud) ✗ ✓ ✗ ✓ ✓
SQLELASTIC
8
Copyright © 2017 NuoDB
ELASTIC SQL ARCHITECTURES
Consider differences in:
+ Hardware requirements
+ Deployment environments
+ Performance results
+ Application expectations
+ Tunability of durability
Approach
Synchronous
Replication
Durable
Distributed Cache
Key Idea
Committing data
transactionally to multiple
locations before returning
Replicating data in memory
on-demand
Topology
Example
Other Elastic SQL DBMS
(Google Cloud Spanner,
CockroachDB)
9
Copyright © 2017 NuoDB
WHAT DOES THIS MEAN…
+ For the business
 Easy migration to cloud
 Continuous application availability
 Reduced licensing, hardware, and operational costs
+ For the architect
 NoSQL scale out for SQL applications
 No need to re-architect existing applications
 Less complexity data management environments
+ For the developer
 Reuse SQL skills
 Focus on new features development versus data management logic
10
Copyright © 2017 NuoDB
NuoDB: MODERN ARCHITECTURE
Fast in-memory
transaction
processing
with dynamic
application data
affinity
Durable storage
management with
scale-out storage
NuoDB Architecture
Legacy RDBMS
Architecture
Storage Mgmt
Query Processing
Modern distributed, peer-to-peer
services acting as a single, logical database
+ New modern database architecture
with independent database services
+ Services scale independently
and elastically
+ Straightforward ANSI SQL interface
+ Deploy across containers,
clouds & data centers
+ Continuous availability for planned
(upgrades) and unplanned outages
11
Copyright © 2017 NuoDB
NuoDB PEER-TO-PEER,
ELASTIC ARCHITECTURE
Durability
SQL Front End
SQL Optimizer
Transaction Handling
Object Caching
Object Coordination
Transaction Engine (TE)
Storage Manager (SM)
All Process Nodes
Available node not yet
provisioned to database
+ All peers have object caching and
coordination
+ Architecture enables on-demand,
elastic scaling
+ Natural resilience supports
continuous availability
12
Copyright © 2017 NuoDB
SM ARCHITECTURETE ARCHITECTURE
13
Copyright © 2017 NuoDB
ATOMS
+ All objects (data and
metadata) are represented
by ATOMs
+ In-memory and on-disk
+ Exist on multiple servers
+ Conflict free replicated
data typdes (CRDT) with
guaranteed ordering
+ Optimized replication
Table ATOM Catalog ATOM
Master Catalog
ATOM
Database ATOM Schema ATOM Sequence ATOMTransaction
Manager ATOM
Index ATOMBlob State ATOM Record State
ATOM
Data ATOM
14
Copyright © 2017 NuoDB
NuoDB ARCHITECTURE: SCALE OUT
Available processes
for scale out
ApplicationA Storage Manager (SM)SM Transaction Engine (TE)TE
1. Initial Database 2. Increased Application Load 3. Provision Available Resource as TE
TE
A A
TE
SMSM
TE
A A
TE
SMSM
A
TETE
A A
TE
SMSM
A
15
Copyright © 2017 NuoDB
NuoDB ARCHITECTURE: RESILIENCE
Available processes
for scale out
ApplicationA Storage Manager (SM)SM Transaction Engine (TE)TE
1. Working Database 2. TE node failure 3. Application reconnects to surviving TE
TETE
A A
TE
SMSM
A
TETE
A A
TE
SMSM
A
TETE
A A
TE
SMSM
A
16
Copyright © 2017 NuoDB
TYPICAL DEPLOYMENT SCENARIOS
Fully Redundant Scale-Out
Running NuoDB on four hosts (2
TEs and 2 SMs) provides a fully
redundant deployment of NuoDB
that can survive any host failing.
To increase throughput, simply add
TEs/SMs to other available
machines on demand.
Warm DR
Minimal DR protection requires a single SM in a
DR site to maintain a fully redundant copy of the
database.
TE
A A
TE
SMSM
Primary DC
SM
DR Site
TE
A A
TE
SMSM
Primary DC
TE
A A
TE
SMSM
A
TE
A
TE
Primary DC
17
Copyright © 2017 NuoDB
TYPICAL DEPLOYMENT SCENARIOS
Hot DR Active-Active
Deploying and running both TEs and SMs in
the DR site provides zero downtime failover
protection for applications.
Deploying applications in the remote site
provides full active-active (read/write)
transactional processing for applications.
Active-Active-Active
TE
A A
TE
SMSM
Primary DC
TE TE
SMSM
DR Site
TE
A A
TE
SMSM
Availability Zone 1
SMSM
Availability Zone 2
TE
A A
TE
SM
Availability Zone 3
TE TE
A A
SM
Increase utilization
and protection with
active-active-active
deployments
18
Copyright © 2017 NuoDB
MULTI-DC SUPPORT & CAP
+ NuoDB preserves consistency during network partition
 Automatically shuts down minority
TE
A A
TE
SMSM
Availability Zone 1
SMSM
Availability Zone 2
TE
A A
TE
A
TE
19
SCALE OUT DEMO
www.nuodb.com/full-demo
End-to-end Asset Finance Software Platform
CHALLENGE
+ Migrating industry-leading
application to SaaS model
+ Lack of cloud offering
inhibiting growth
+ MySQL couldn’t scale far
enough; Oracle licensing
on AWS too expensive
WHY NuoDB
+ Designed for elastic cloud
scalability
+ Continuous availability
with active-active
architecture
+ Maintains transactional
integrity
BENEFITS
+ Strong ROI for Alfa – and
their clients
+ Reduce infrastructure cost
by 90% over Oracle
+ Increase application
availability and minimize
response time
Copyright © 2017 NuoDB
NuoDB
+ Founded in 2010
+ Management team and investors
are database pioneers
+ Patented elastic SQL database
+ Strong Gartner validation
+ Leading SaaS & on-premises
applications run on NuoDB
+ HQ in Cambridge, MA
22
Copyright © 2017 NuoDB
SUMMARY
+ Application architectures have evolved
+ Relational databases remain locked in the past
 Single monolithic process
 Requires special considerations to scale performance
+ Elastic SQL databases
 Scale out and availability benefits of NoSQL
 No sacrificing the advantages of SQL databases
23
Copyright © 2017 NuoDB
THANK YOU!
The Elastic SQL Database facebook.com/nuodb/
@NuoDB
youtube.com/nuoDB
linkedin.com/company/nuodb
github.com/nuodb
NuoDB.com
FOLLOW US:
24

Elastic SQL Database: Oxymoron or Emerging Reality? (Database Month, June 2017)

  • 1.
    Ariff Kassam, VPof Products – akassam@nuodb.com Christina Wong, Dir. of Product Marketing – cwong@nuodb.com ELASTIC SQL: FROM OXYMORON TO EMERGING REALITY June 23, 2017
  • 2.
    Copyright © 2017NuoDB ELASTIC SQL DATABASE + Traditional SQL databases are not suited for modern system architectures or development practices  Doesn’t fit with scale out, container, dynamic orchestration architectures + NoSQL databases better suited for new applications but hard to migrate existing applications + Market need: Provide scale out, containerization, HA for 1
  • 3.
    Copyright © 2017NuoDB ARCHITECTURE EVOLUTION Mainframe Presentation App Logic Data Mgmt Monolithic Open Systems Presentation App Logic Data Mgmt Client / Server Presentation App Logic Data Mgmt N-Tier Internet Cloud Microservices Data Mgmt Infrastructure Architectures Application Architectures 2
  • 4.
    Copyright © 2017NuoDB DATABASE ARCHITECTURES + Relational database architectures have not changed significantly with infrastructure advances  Hardware: Scale up or shared disk  Application: 2 Phase Commit or Sharding 3 Scale Up Shared Disk 2 Phase Commit Sharded Database
  • 5.
    Copyright © 2017NuoDB WHAT ABOUT NOSQL? + Provides scale out with automatic sharding + Can be an option for new applications or existing applications with significant re-write + Challenging trade-offs  Account for eventual consistency  Code for transactional semantics  Code for certain queries (joins, etc) with static logic  Manage and maintain data structures 4
  • 6.
    Copyright © 2017NuoDB TODAY’S DATABASE MARKET PROBLEM Elastic SQL Elastic SQL: The benefits of Traditional Databases & The Cloud SQL Logic ACID: data consistency Zero Downtime Elastic scale-out SQL database abstraction Virtualization, Commodity & Cloud Traditional RDBMS NoSQL MARKET GAP SQL Logic ACID: data consistency SQL database abstraction Zero Downtime Elastic scale-out Virtualization, Commodity & Cloud 5
  • 8.
    Copyright © 2017NuoDB ELASTIC SQL DATABASE An elastic SQL database combines the scale-out simplicity, elasticity, and continuous availability that cloud applications require with the transactional consistency and durability that databases of record demand. 7
  • 9.
    Copyright © 2017NuoDB THE EMERGING ELASTIC SQL MARKET Traditional RDBMS NoSQL Elastic SQL Business Database of Record ✓ ✗ ✓ ✓ ✓ ANSI-Standard SQL ✓ ✗ (✓) (✓) ✓ Migrate existing SQL apps ✓ ✗ ✗ ✗ ✓ In-memory performance ✓ ✓ ✓ ✗ ✓ Simple elastic scale-out/in (reads and writes) ✗ ✓ ✓ ✓ ✓ Zero downtime ✗ ✓ ✓ ✓ ✓ Deployment flexibility (on-prem, private or public cloud) ✗ ✓ ✗ ✓ ✓ SQLELASTIC 8
  • 10.
    Copyright © 2017NuoDB ELASTIC SQL ARCHITECTURES Consider differences in: + Hardware requirements + Deployment environments + Performance results + Application expectations + Tunability of durability Approach Synchronous Replication Durable Distributed Cache Key Idea Committing data transactionally to multiple locations before returning Replicating data in memory on-demand Topology Example Other Elastic SQL DBMS (Google Cloud Spanner, CockroachDB) 9
  • 11.
    Copyright © 2017NuoDB WHAT DOES THIS MEAN… + For the business  Easy migration to cloud  Continuous application availability  Reduced licensing, hardware, and operational costs + For the architect  NoSQL scale out for SQL applications  No need to re-architect existing applications  Less complexity data management environments + For the developer  Reuse SQL skills  Focus on new features development versus data management logic 10
  • 12.
    Copyright © 2017NuoDB NuoDB: MODERN ARCHITECTURE Fast in-memory transaction processing with dynamic application data affinity Durable storage management with scale-out storage NuoDB Architecture Legacy RDBMS Architecture Storage Mgmt Query Processing Modern distributed, peer-to-peer services acting as a single, logical database + New modern database architecture with independent database services + Services scale independently and elastically + Straightforward ANSI SQL interface + Deploy across containers, clouds & data centers + Continuous availability for planned (upgrades) and unplanned outages 11
  • 13.
    Copyright © 2017NuoDB NuoDB PEER-TO-PEER, ELASTIC ARCHITECTURE Durability SQL Front End SQL Optimizer Transaction Handling Object Caching Object Coordination Transaction Engine (TE) Storage Manager (SM) All Process Nodes Available node not yet provisioned to database + All peers have object caching and coordination + Architecture enables on-demand, elastic scaling + Natural resilience supports continuous availability 12
  • 14.
    Copyright © 2017NuoDB SM ARCHITECTURETE ARCHITECTURE 13
  • 15.
    Copyright © 2017NuoDB ATOMS + All objects (data and metadata) are represented by ATOMs + In-memory and on-disk + Exist on multiple servers + Conflict free replicated data typdes (CRDT) with guaranteed ordering + Optimized replication Table ATOM Catalog ATOM Master Catalog ATOM Database ATOM Schema ATOM Sequence ATOMTransaction Manager ATOM Index ATOMBlob State ATOM Record State ATOM Data ATOM 14
  • 16.
    Copyright © 2017NuoDB NuoDB ARCHITECTURE: SCALE OUT Available processes for scale out ApplicationA Storage Manager (SM)SM Transaction Engine (TE)TE 1. Initial Database 2. Increased Application Load 3. Provision Available Resource as TE TE A A TE SMSM TE A A TE SMSM A TETE A A TE SMSM A 15
  • 17.
    Copyright © 2017NuoDB NuoDB ARCHITECTURE: RESILIENCE Available processes for scale out ApplicationA Storage Manager (SM)SM Transaction Engine (TE)TE 1. Working Database 2. TE node failure 3. Application reconnects to surviving TE TETE A A TE SMSM A TETE A A TE SMSM A TETE A A TE SMSM A 16
  • 18.
    Copyright © 2017NuoDB TYPICAL DEPLOYMENT SCENARIOS Fully Redundant Scale-Out Running NuoDB on four hosts (2 TEs and 2 SMs) provides a fully redundant deployment of NuoDB that can survive any host failing. To increase throughput, simply add TEs/SMs to other available machines on demand. Warm DR Minimal DR protection requires a single SM in a DR site to maintain a fully redundant copy of the database. TE A A TE SMSM Primary DC SM DR Site TE A A TE SMSM Primary DC TE A A TE SMSM A TE A TE Primary DC 17
  • 19.
    Copyright © 2017NuoDB TYPICAL DEPLOYMENT SCENARIOS Hot DR Active-Active Deploying and running both TEs and SMs in the DR site provides zero downtime failover protection for applications. Deploying applications in the remote site provides full active-active (read/write) transactional processing for applications. Active-Active-Active TE A A TE SMSM Primary DC TE TE SMSM DR Site TE A A TE SMSM Availability Zone 1 SMSM Availability Zone 2 TE A A TE SM Availability Zone 3 TE TE A A SM Increase utilization and protection with active-active-active deployments 18
  • 20.
    Copyright © 2017NuoDB MULTI-DC SUPPORT & CAP + NuoDB preserves consistency during network partition  Automatically shuts down minority TE A A TE SMSM Availability Zone 1 SMSM Availability Zone 2 TE A A TE A TE 19
  • 21.
  • 22.
    End-to-end Asset FinanceSoftware Platform CHALLENGE + Migrating industry-leading application to SaaS model + Lack of cloud offering inhibiting growth + MySQL couldn’t scale far enough; Oracle licensing on AWS too expensive WHY NuoDB + Designed for elastic cloud scalability + Continuous availability with active-active architecture + Maintains transactional integrity BENEFITS + Strong ROI for Alfa – and their clients + Reduce infrastructure cost by 90% over Oracle + Increase application availability and minimize response time
  • 23.
    Copyright © 2017NuoDB NuoDB + Founded in 2010 + Management team and investors are database pioneers + Patented elastic SQL database + Strong Gartner validation + Leading SaaS & on-premises applications run on NuoDB + HQ in Cambridge, MA 22
  • 24.
    Copyright © 2017NuoDB SUMMARY + Application architectures have evolved + Relational databases remain locked in the past  Single monolithic process  Requires special considerations to scale performance + Elastic SQL databases  Scale out and availability benefits of NoSQL  No sacrificing the advantages of SQL databases 23
  • 25.
    Copyright © 2017NuoDB THANK YOU! The Elastic SQL Database facebook.com/nuodb/ @NuoDB youtube.com/nuoDB linkedin.com/company/nuodb github.com/nuodb NuoDB.com FOLLOW US: 24

Editor's Notes

  • #5 However, one layer of application architectures has not evolved as fast or as far as the other application layers - that’s the Data Access layer. More specifically, relational databases haven’t adapted to infrastructure improvements Performance improvements required scale up by increase cpu, memory and disk capacity Certain database vendors have advanced memory sharing architectures to enable limited scale out Only proven solution requires application changes to shard the database into smaller databases - well known challenges with database sharding
  • #23 (add notes to all case study slides)