Introducing MariaDB
Platform X3 and the rise
of hybrid everything
Shane K Johnson
Senior Director of Product Marketing
MariaDB Corporation
Agenda
1. Hybrid workloads
2. MariaDB Platform X3
3. Scalability
4. Real-world use cases
5. Hybrid cloud
6. Database consolidation
7. Getting started with Docker
8. Managed Service
Hybrid workloads
Database workloads
Transactional
Current data
Range queries
Known queries
Historical data
Aggregate queries
Unkown queries
Analytical
Analytical Transactional
Performance
Range
Analytical Transactional
Performance
Range
Analytical Transactional
Performance
Range
More data
More customers
AX TX
Performance
Range
Database (OLTP)
AX TX
Performance
Range
Data warehouse (OLAP)
Database workloads
Transactional
Current data
Range queries
Known queries
Row-based storage
Indexes
Clustered/Replicated
Historical data
Aggregate queries
Unkown queries
Columnar storage
No indexes
Distributed
Analytical
AX TX
Performance
Range
Database (OLTP)
AX TX
Performance
Range
Data warehouse (OLAP)
Application development BI/reporting + data science
Application
(eCommerce)
Transactional
Show me all new products in
the science fiction category
Analytical
Show me the top products
added to shopping carts or
purchased today, and with
low inventory.
Actionable insight
I should buy one now
because everyone wants
one, and they’ll be sold out
by the end of the day!
Application
(banking)
Transactional
Show me all pending
transactions
Analytical
Show me when my balance
will run low based on
historical transactions and
my current spending rate
Actionable insight
I should transfer some
money from my savings
account to my checking
account until I get paid again!
Application
(pay-per-click ads)
Transactional
Create campaign
Create ad group
Create ad
Capture impressions
Capture clicks
Capture charges
Analytical
Show me which ads will
perform worse based past
performance and changes in
keyword costs and searches
Actionable insight
I should pause this ad and
raise the budget on others in
order to meet my goals!
Data warehouse
(OLAP)
Database
(OLTP)
Hybrid workloads: the problem
Transactions
App/dev
Analytics
BI/reporting + data science
Data warehouse
(OLAP)
Database
(Hybrid)
Hybrid workloads: the solution
Transactions Analytics
App/dev
Analytics
BI/reporting + data science
MariaDB Platform X3
MariaDB TX
MariaDB Server
MariaDB MaxScale
InnoDB/MyRocks
MariaDB AX
MariaDB Server
MariaDB MaxScale
ColumnStore
App/dev
MariaDB
TX
BI/data science
MariaDB
AX
MariaDB TX 3.0
MariaDB Server 10.3
MariaDB MaxScale 2.2
InnoDB/MyRocks
MariaDB AX 2.0
MariaDB Server 10.2
MariaDB MaxScale 2.2
ColumnStore 1.2
MariaDB Platform X3
MariaDB MaxScale 2.3
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Application development
MariaDB Platform X3
MariaDB MaxScale 2.3
CDC
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Transactional Analytical
The database proxy inspects queries and routes them to transactional
and/or analytical database instances.
MariaDB Platform X3
MariaDB MaxScale 2.3
CDC
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Transactional Analytical
The database proxy inspects queries and routes them to transactional
and/or analytical database instances.
The change-data-capture stream replicates all writes from transactional
databases to analytical databases within microbatches.
MariaDB Platform X3
MariaDB MaxScale 2.3
CDC
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Transactional Analytical
Containers
MariaDB Platform X3
MariaDB MaxScale 2.3
CDC
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Transactional Analytical
Kubernetes (Helm) Docker (Compose)
Import bulk data
Containers
MariaDB Platform X3
MariaDB MaxScale 2.3
CDC
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Transactional Analytical
Kubernetes (Helm) Docker (Compose)
Spark connector
C/Java/Python API
Ingest streaming data
Kafka connector
Applications
Containers
MariaDB Platform X3
MariaDB MaxScale 2.3
CDC
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Transactional Analytical
Kubernetes (Helm) Docker (Compose)
C JDBC ODBC Node.js
Ingest streaming data
Kafka connector
Import bulk data
Spark connector
C/Java/Python API
Applications
Containers
MariaDB Platform X3
MariaDB MaxScale 2.3
CDC
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Transactional Analytical
Kubernetes (Helm) Docker (Compose)
C JDBC ODBC Node.js
Ingest streaming data
Kafka connector
Administration
SQL Diagnostic
Manager
SQLyog
MariaDB Backup
MariaDB Flashback
Import bulk data
Spark connector
C/Java/Python API
Scalability
Hybrid workloads: why scalability is needed
Applications have transactional
and analytical queries
1. Constrained by limited,
lightweight analytics
2. Need full analytics to
create competitive features
Outgrowing OLTP
Applications with lots of
customers, lots of transactions
1. Limited to current or recent
transaction data (months)
2. Need access to all
historical data (years)
Using historical data
SaaS customers are becoming
data-driven organizations
1. They don’t have access to
their own data
2. They need to analyze it in
unknown/unexpected ways
Exposing analytics
Hybrid workloads: options
● Oracle
○ In-Memory Column Store
● Microsoft SQL Server
○ ColumnStore Indexes (Clustered or Nonclustered)
● IBM Db2
○ Shadow Tables
● MySQL Enterprise
○ None
● EnterpriseDB Postgres Platform
○ None
Hybrid workloads: full comparison
* IBM Db2 Shadow Tables, ** Oracle IM column store can read from on-disk row storage if needed
Oracle Microsoft IBM * MariaDB MySQL Postgres
Row storage (OLTP) Yes Yes Yes Yes Yes Yes
Sharded Yes No Yes Yes No No
Columnar storage (OLAP) Yes Yes Yes Yes No No
Disk-based No ** Yes Yes Yes - -
Distributed Yes (RAC) No No Yes - -
Microsoft
SQL Server
Columnar storage
Row storage
Microsoft
SQL Server
Row storage
Columnar storage
Application
Connection 1 Connection 2
Transactional Analytical
Application
Connection 1
MariaDB
Server
Row storage
MariaDB Server
Columnar
storage
MariaDB
MaxScale
Transactional Analytical
MariaDB Server
(Spider)
Application
Connection 1
MariaDB
Server 1
Row storage
MariaDB
Server 2
Row storage
MariaDB
Server n
Row storage
MariaDB Server
(ColumnStore)
Sharding
MariaDB
MaxScale
Transactional Analytical
MariaDB Server
Columnar
storage
MariaDB Server
(Spider)
Application
Connection 1
MariaDB Server
(ColumnStore)
Node 1
Columnar
storage
Node 2
Columnar
storage
Node n
Columnar
storage
Distributed storage
MariaDB
MaxScale
Transactional Analytical
MariaDB
Server
Row storage
MariaDB Server
(Spider)
Application
Connection 1
MariaDB
Server 1
Row storage
MariaDB
Server 2
Row storage
MariaDB
Server n
Row storage
MariaDB Server
(ColumnStore)
CS node 1
Columnar
storage
CS node 2
Columnar
storage
CS node n
Columnar
storage
Sharding Distributed storage
MariaDB
MaxScale
Transactional Analytical
Real-world use cases
Retail (market research)
Database
(Hybrid)
Transactions Analytics
Transactional
Capture daily product prices
Update product information
Show current prices
Store current pricing data
Analytical
Show prices over time
Self-service analytics
Store historical pricing data
Telecommunications (IP telephony – SaaS)
Database
(Hybrid)
Transactions Analytics
Transactional
Capture call detail records
Charge by call/message
Generate bills
Analytical
Monitor usage
Identify peak periods
Estimate costs
Self-service analytics
Finance (trading)
Database
(Hybrid)
Transactions Analytics
Transactional
Capture trades
Analytical
Analyze trading data
Archiving for regulator compliance
(historical data)
Self-service analytics
Hybrid cloud
Hybrid workloads: perfect for hybrid cloud
● MariaDB Platform separates and isolates different workloads
○ Run different workloads on different infrastructure
○ Place different workloads closer to different users
○ Scale different workloads on different hardware
MariaDB Platform
MariaDB Server
MariaDB Server
MariaDB Server
MariaDB Server
MariaDB Server
InnoDB/MyRocks
MariaDB Server
ColumnStore
On premises
for transactions
Keep transactions
close to the business
Public cloud
for analytics
Incremental path to
the cloud, places
analytics close to
customers
MariaDB Platform
On premises
for analytics
Place analytics closer
to employees,
aggregate all
transactions
Public cloud
for transactions
Place transactions
close to customers,
geographically
distributed
MariaDB Server
MariaDB Server
MariaDB Server
ColumnStore
MariaDB Server
MariaDB Server
MariaDB Server
InnoDB/MyRocks
Database
consolidation
Service #2
Database #1
RDBMS, transactions
(Oracle)
Database #2
NoSQL, scalability
(MongoDB)
Database #3
Columnar, analytics
(Vertica)
Purchases Carts Clickstream
Service #1 Service #3
MariaDB Platform
Carts
MariaDB Server MariaDB Server MariaDB Server
Purchases Clickstream
MyRocks Spider
Shard 2Shard 1 Shard 3
ColumnStore
MariaDB MaxScale
Service #2Service #1 Service #3
Getting started with
Docker
MariaDB Platform X3 in a container
● MariaDB Platform “in-a-box”
● Fully-configured and ready to use out of the box
● Intended as a quickstart for development
● Launched with a Docker command
● Runs on a laptop
https://github.com/mariadb-corporation/mariadb-platform-docker/tree/master/single-container
MariaDB Platform X3 in a container
MariaDB Platform
Managed Service
MariaDB Platform Managed Service
Provision
Install
Configure
Optimize
Recover
Maintain
Support
Reference architecture
Disaster recovery
Security audits
Performance tuning
Query optimization
Schema changes
Database upgrades
Database migration
Techical support
Consultative support
Cookiecutter
Limited features
Little to no support
Delayed upgrades
Downtime
AWS RDS/Aurora MariaDB Platform
Managed Service
MariaDB vs. Amazon
AWS RDS AWS Aurora MariaDB Platform
Hybrid cloud Preview No Yes
Hybrid workloads No Limited Yes
Proactive care No No Yes
Latest version No No Yes
Technical support No No Yes
Consultative support No No Yes
Security fixes No No Yes
MariaDB vs. Amazon
Transaction replay and causal reads too!
AWS RDS AWS Aurora MariaDB Platform
Multi-master No Preview Yes
Oracle compatibility Yes No Yes
Temporal Yes No Yes
Database firewall No No Yes
Data masking No No Yes
Change-data-capture No No Yes
Kafka connector No No Yes
THANK YOU!

Improving Transactional Applications with Analytics

  • 1.
    Introducing MariaDB Platform X3and the rise of hybrid everything Shane K Johnson Senior Director of Product Marketing MariaDB Corporation
  • 2.
    Agenda 1. Hybrid workloads 2.MariaDB Platform X3 3. Scalability 4. Real-world use cases 5. Hybrid cloud 6. Database consolidation 7. Getting started with Docker 8. Managed Service
  • 3.
  • 4.
    Database workloads Transactional Current data Rangequeries Known queries Historical data Aggregate queries Unkown queries Analytical
  • 5.
  • 6.
  • 7.
    AX TX Performance Range Database (OLTP) AXTX Performance Range Data warehouse (OLAP)
  • 8.
    Database workloads Transactional Current data Rangequeries Known queries Row-based storage Indexes Clustered/Replicated Historical data Aggregate queries Unkown queries Columnar storage No indexes Distributed Analytical
  • 9.
    AX TX Performance Range Database (OLTP) AXTX Performance Range Data warehouse (OLAP) Application development BI/reporting + data science
  • 10.
    Application (eCommerce) Transactional Show me allnew products in the science fiction category Analytical Show me the top products added to shopping carts or purchased today, and with low inventory. Actionable insight I should buy one now because everyone wants one, and they’ll be sold out by the end of the day!
  • 11.
    Application (banking) Transactional Show me allpending transactions Analytical Show me when my balance will run low based on historical transactions and my current spending rate Actionable insight I should transfer some money from my savings account to my checking account until I get paid again!
  • 12.
    Application (pay-per-click ads) Transactional Create campaign Createad group Create ad Capture impressions Capture clicks Capture charges Analytical Show me which ads will perform worse based past performance and changes in keyword costs and searches Actionable insight I should pause this ad and raise the budget on others in order to meet my goals!
  • 13.
    Data warehouse (OLAP) Database (OLTP) Hybrid workloads:the problem Transactions App/dev Analytics BI/reporting + data science
  • 14.
    Data warehouse (OLAP) Database (Hybrid) Hybrid workloads:the solution Transactions Analytics App/dev Analytics BI/reporting + data science
  • 15.
  • 16.
    MariaDB TX MariaDB Server MariaDBMaxScale InnoDB/MyRocks MariaDB AX MariaDB Server MariaDB MaxScale ColumnStore App/dev MariaDB TX BI/data science MariaDB AX
  • 17.
    MariaDB TX 3.0 MariaDBServer 10.3 MariaDB MaxScale 2.2 InnoDB/MyRocks MariaDB AX 2.0 MariaDB Server 10.2 MariaDB MaxScale 2.2 ColumnStore 1.2 MariaDB Platform X3 MariaDB MaxScale 2.3 MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3
  • 18.
    Application development MariaDB PlatformX3 MariaDB MaxScale 2.3 CDC MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3 Transactional Analytical
  • 19.
    The database proxyinspects queries and routes them to transactional and/or analytical database instances. MariaDB Platform X3 MariaDB MaxScale 2.3 CDC MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3 Transactional Analytical
  • 20.
    The database proxyinspects queries and routes them to transactional and/or analytical database instances. The change-data-capture stream replicates all writes from transactional databases to analytical databases within microbatches. MariaDB Platform X3 MariaDB MaxScale 2.3 CDC MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3 Transactional Analytical
  • 21.
    Containers MariaDB Platform X3 MariaDBMaxScale 2.3 CDC MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3 Transactional Analytical Kubernetes (Helm) Docker (Compose)
  • 22.
    Import bulk data Containers MariaDBPlatform X3 MariaDB MaxScale 2.3 CDC MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3 Transactional Analytical Kubernetes (Helm) Docker (Compose) Spark connector C/Java/Python API Ingest streaming data Kafka connector
  • 23.
    Applications Containers MariaDB Platform X3 MariaDBMaxScale 2.3 CDC MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3 Transactional Analytical Kubernetes (Helm) Docker (Compose) C JDBC ODBC Node.js Ingest streaming data Kafka connector Import bulk data Spark connector C/Java/Python API
  • 24.
    Applications Containers MariaDB Platform X3 MariaDBMaxScale 2.3 CDC MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3 Transactional Analytical Kubernetes (Helm) Docker (Compose) C JDBC ODBC Node.js Ingest streaming data Kafka connector Administration SQL Diagnostic Manager SQLyog MariaDB Backup MariaDB Flashback Import bulk data Spark connector C/Java/Python API
  • 25.
  • 26.
    Hybrid workloads: whyscalability is needed Applications have transactional and analytical queries 1. Constrained by limited, lightweight analytics 2. Need full analytics to create competitive features Outgrowing OLTP Applications with lots of customers, lots of transactions 1. Limited to current or recent transaction data (months) 2. Need access to all historical data (years) Using historical data SaaS customers are becoming data-driven organizations 1. They don’t have access to their own data 2. They need to analyze it in unknown/unexpected ways Exposing analytics
  • 27.
    Hybrid workloads: options ●Oracle ○ In-Memory Column Store ● Microsoft SQL Server ○ ColumnStore Indexes (Clustered or Nonclustered) ● IBM Db2 ○ Shadow Tables ● MySQL Enterprise ○ None ● EnterpriseDB Postgres Platform ○ None
  • 28.
    Hybrid workloads: fullcomparison * IBM Db2 Shadow Tables, ** Oracle IM column store can read from on-disk row storage if needed Oracle Microsoft IBM * MariaDB MySQL Postgres Row storage (OLTP) Yes Yes Yes Yes Yes Yes Sharded Yes No Yes Yes No No Columnar storage (OLAP) Yes Yes Yes Yes No No Disk-based No ** Yes Yes Yes - - Distributed Yes (RAC) No No Yes - -
  • 29.
    Microsoft SQL Server Columnar storage Rowstorage Microsoft SQL Server Row storage Columnar storage Application Connection 1 Connection 2 Transactional Analytical
  • 30.
    Application Connection 1 MariaDB Server Row storage MariaDBServer Columnar storage MariaDB MaxScale Transactional Analytical
  • 31.
    MariaDB Server (Spider) Application Connection 1 MariaDB Server1 Row storage MariaDB Server 2 Row storage MariaDB Server n Row storage MariaDB Server (ColumnStore) Sharding MariaDB MaxScale Transactional Analytical MariaDB Server Columnar storage
  • 32.
    MariaDB Server (Spider) Application Connection 1 MariaDBServer (ColumnStore) Node 1 Columnar storage Node 2 Columnar storage Node n Columnar storage Distributed storage MariaDB MaxScale Transactional Analytical MariaDB Server Row storage
  • 33.
    MariaDB Server (Spider) Application Connection 1 MariaDB Server1 Row storage MariaDB Server 2 Row storage MariaDB Server n Row storage MariaDB Server (ColumnStore) CS node 1 Columnar storage CS node 2 Columnar storage CS node n Columnar storage Sharding Distributed storage MariaDB MaxScale Transactional Analytical
  • 34.
  • 35.
    Retail (market research) Database (Hybrid) TransactionsAnalytics Transactional Capture daily product prices Update product information Show current prices Store current pricing data Analytical Show prices over time Self-service analytics Store historical pricing data
  • 36.
    Telecommunications (IP telephony– SaaS) Database (Hybrid) Transactions Analytics Transactional Capture call detail records Charge by call/message Generate bills Analytical Monitor usage Identify peak periods Estimate costs Self-service analytics
  • 37.
    Finance (trading) Database (Hybrid) Transactions Analytics Transactional Capturetrades Analytical Analyze trading data Archiving for regulator compliance (historical data) Self-service analytics
  • 38.
  • 39.
    Hybrid workloads: perfectfor hybrid cloud ● MariaDB Platform separates and isolates different workloads ○ Run different workloads on different infrastructure ○ Place different workloads closer to different users ○ Scale different workloads on different hardware
  • 40.
    MariaDB Platform MariaDB Server MariaDBServer MariaDB Server MariaDB Server MariaDB Server InnoDB/MyRocks MariaDB Server ColumnStore On premises for transactions Keep transactions close to the business Public cloud for analytics Incremental path to the cloud, places analytics close to customers
  • 41.
    MariaDB Platform On premises foranalytics Place analytics closer to employees, aggregate all transactions Public cloud for transactions Place transactions close to customers, geographically distributed MariaDB Server MariaDB Server MariaDB Server ColumnStore MariaDB Server MariaDB Server MariaDB Server InnoDB/MyRocks
  • 42.
  • 43.
    Service #2 Database #1 RDBMS,transactions (Oracle) Database #2 NoSQL, scalability (MongoDB) Database #3 Columnar, analytics (Vertica) Purchases Carts Clickstream Service #1 Service #3
  • 44.
    MariaDB Platform Carts MariaDB ServerMariaDB Server MariaDB Server Purchases Clickstream MyRocks Spider Shard 2Shard 1 Shard 3 ColumnStore MariaDB MaxScale Service #2Service #1 Service #3
  • 45.
  • 46.
    MariaDB Platform X3in a container ● MariaDB Platform “in-a-box” ● Fully-configured and ready to use out of the box ● Intended as a quickstart for development ● Launched with a Docker command ● Runs on a laptop https://github.com/mariadb-corporation/mariadb-platform-docker/tree/master/single-container
  • 47.
    MariaDB Platform X3in a container
  • 48.
  • 49.
    MariaDB Platform ManagedService Provision Install Configure Optimize Recover Maintain Support Reference architecture Disaster recovery Security audits Performance tuning Query optimization Schema changes Database upgrades Database migration Techical support Consultative support Cookiecutter Limited features Little to no support Delayed upgrades Downtime AWS RDS/Aurora MariaDB Platform Managed Service
  • 50.
    MariaDB vs. Amazon AWSRDS AWS Aurora MariaDB Platform Hybrid cloud Preview No Yes Hybrid workloads No Limited Yes Proactive care No No Yes Latest version No No Yes Technical support No No Yes Consultative support No No Yes Security fixes No No Yes
  • 51.
    MariaDB vs. Amazon Transactionreplay and causal reads too! AWS RDS AWS Aurora MariaDB Platform Multi-master No Preview Yes Oracle compatibility Yes No Yes Temporal Yes No Yes Database firewall No No Yes Data masking No No Yes Change-data-capture No No Yes Kafka connector No No Yes
  • 52.