OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

NoSQL & MySQL: The Best of
Both Worlds
Andrew Morgan (@andrewmorgan)
www.clusterdb.com
Principal MySQL Product Manager
11th June 2014

Copyright © 2014, Oracle and/or its affiliates. All rights Reserved.2
Safe Harbour Statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole
discretion of Oracle.

MySQL Customers
World’s Most Popular Open Source Database
WebSaaS, Hosting
Enterprise OEM / ISV’s
Telecom

Session Agenda
 NoSQL – What are people looking for?
 RDBMS – What advantages do they still have?
 How MySQL Delivers the Best of Both Worlds
– MySQL Cluster
 NoSQL attributes: Scale-out, performance, ease-of-use, schema
flexibility, on-line operations
 NoSQL APIs
– Key-Value store access to InnoDB (Memcached)

Types of NoSQL stores
Key-Value
• Cassandra
• Memcached
• BigTable
• Hadoop
• Voldermort
Document
• MongoDB
• CouchDB
Graph
• Neo4J
• FlockDB

Graph Database

What NoSQL must deliver
 Massive scalability
– No application-level sharding
 Performance
 High Availability/Fault Tolerance
 Ease of use
– Simple operations/administration
– Simple APIs
– Quickly evolve application &
schema
Scalability
Performance
HA
Ease of use

Still a role for the SQL (RDBMS)?
NoSQL
Simple access patterns
Compromise on consistency
for performance
Ad-hoc data format
Simple operation
SQL
Complex queries with joins
ACID transactions
Well defined schemas
Rich set of tools
 No best single solution
fits all
 Mix and match
Scalability
Performance
HA
Ease of use
SQL/Joins
ACID
Transactions

The Temptation to Jettison Relational Model
 Relational Model:
– Data stored in multiple
tables
– Many lookups to retrieve a
user’s timeline
– Is it worth the effort in
setting up this complex
data model?
 Simpler just to store as one
document?
The allure of document stores
Examples borrowed from @sarahmei https://speakerdeck.com/sarahmei/switching-data-stores-a-postmodern-comedy
user
friend
post
comment
like
liker
commenter
many
many
many
one
one
many

 Document Model:
– Entire timeline in a single
document (row)
– Single lookup to retrieve the
user’s timeline
– Brilliantly efficient model when
the document truly contains
self-contained information
 Like a real-world document!
{name: ‘Joe’,
url: ‘…’
stream:[
{friend:{
name: ‘Jane’,
posts:[{content: ‘…’,
comments:[
{comment: ‘…’,
commenter: ‘Joe’},
{…}
],
likes: [‘Joe’, ‘Fred’, ‘Billy’]
},
{…},{…},…
]
}, {…},{…},…
}

 These are all people who
have their own data that
users will want to view or
click through:
– Name
– url
– Timeline
 Easy to represent with FKs
in a relational model
But when the data isn’t self contained…
user
friend
post
comment
like
liker
commenter
many
many
many
one
one
many

 Do I store all data for all
friends and likers again at
every point they appear in the
document?
– Massive amount of repeated
data
 Wasted space
 Needs to be kept in sync (and
you don’t have transactions to
provide consistency
{name: ‘Joe’,
url: ‘…’
stream:[
{friend:{
name: ‘Jane’,
comments:[
commenter: ‘Joe’},
{…}
],
likes: [‘Joe’, ‘Fred’, ‘Billy’]
},
{…},{…},…
]
}, {…},{…},…
}

{name: 83746251,
url: ‘…’
stream:[
{friend:{
name: 9384726153,
comments:[
commenter: 83746251},
{…}
],
likes: [83746251, 750730283,
2938493820]
},
{…},{…},…
]
}, {…},{…},…
}
 The reality is that the
developer will store the user-
ids instead
 Developer is responsible for
implementing ‘joins’ in the
application
– e.g. scanning all timelines for
where a user made a
comment or liked a post
 No transactions to ensure
consistency between
documents

MySQL Cluster: Overview
• Auto-Sharding, Multi-Master
• ACID Compliant, OLTP + Real-Time Analytics
HIGH SCALE,
READS + WRITES
• Shared nothing, no Single Point of Failure
• Self Healing + On-Line Operations
99.999%
AVAILABILITY
• Open Source + Commercial Editions
• Commodity hardware + Management, Monitoring Tools
LOW TCO
• Key/Value + Complex, Relational Queries
• SQL + Memcached + JavaScript + Java + JPA + HTTP/REST & C++
SQL + NoSQL
• In-Memory Optimization + Disk-Data
• Predictable Low-Latency, Bounded Access Time
REAL-TIME

Who’s Using MySQL Cluster?

MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Clients
Application Layer
Management

Data Layer
Clients
Application Layer
Management
Management

Data Layer
Application Layer
Management
Management
Clients

On-line Operations
 Scale the cluster (add & remove nodes on-line)
 Repartition tables
 Upgrade / patch servers & OS
 Upgrade / patch MySQL Cluster
 Back-Up
 Evolve the schema on-line, in real-time

Scale-Out Reads & Writes on Commodity Hardware
 8 x Commodity Intel Servers
– 2 x 6-core processors 2.93GHz
– x5670 processors (24 threads)
– 48GB RAM
 Infiniband networking
 flexAsynch benchmark (NDB API)
0
200
400
600
800
1,000
1,200
2 4 8
Millions
Number of Data Nodes
SELECT Queries per Minute
0
20
40
60
80
100
120
4 8
Millions
Number of Data Nodes
UPDATE Queries per Minute

JOIN Performance
33K rows over 11
tables
Must Analyze tables
for best results
mysql> ANALYZE TABLE <tab-name>;
0
10
20
30
40
50
60
70
80
90
100
MySQL Cluster 7.1 MySQL Cluster 7.2
Query Execution Time Seconds
Scalability a
Performance a
HA a
Ease of use
SQL/Joins a
ACID
Transactions
a
70x
More
performance

MySQL Cluster 7.4
 Performance gain over 7.3
– 47% (Read-Only)
– 38% (Read-Write)
Better performance and operational simplicity
labs.mysql.com
 Faster node restarts
– Recovering nodes rejoin the cluster
faster

Creating & running your first Cluster
The traditional way (pre-MCM & Auto-Installer)
Download &
Extract
•edelivery.oracle.com
•www.mysql.com
•dev.mysql.com
Configure
•Cluster-wide
“config.ini”
•Per-mysqld “my.cnf”
Start processes
•Management Nodes
•Data Nodes
•MySQL Servers

Scalability a
Performance a
HA a
Ease of use a
SQL/Joins a
ACID
Transactions
a

MySQL Cluster 7.3: Auto-Installer
 Fast configuration
 Auto-discovery
 Workload optimized
 Repeatable best
practices
Specify
Workload
Auto-
Discover
Define
TopologyDeploy

MySQL Cluster Manager
1. Download MCM/Cluster package from edelivery.oracle.com:
2. Unzip
3. Run agent, define, create & start Cluster!
$> binmcmd --bootstrap
MySQL Cluster Manager 1.1.2 started
Connect to MySQL Cluster Manager by running "D:AndrewDocumentsMySQLmcmbinmcm" -a NOVA:1862
Configuring default cluster 'mycluster'...
Starting default cluster 'mycluster'...
Cluster 'mycluster' started successfully
ndb_mgmd NOVA:1186
ndbd NOVA
ndbd NOVA
mysqld NOVA:3306
mysqld NOVA:3307
ndbapi *
Connect to the database by running "D:AndrewDocumentsMySQLmcmclusterbinmysql" -h NOVA -P 3306 -u root

MCM: Upgrade Cluster
mysql> upgrade cluster
--package=7.3 mycluster;

Scalability a
Performance a
HA a
Ease of use a
SQL/Joins a
ACID
Transactions
a

NoSQL Access to MySQL Cluster data
ClusterJ
MySQL
JDBC
Apps
JPA
JNI
Python Ruby
ClusterJPA
Apps Apps Apps Apps Apps
Node.js
JS
Apps
mod-ndb
Apache
Apps
ndb-eng
Memcached
Apps Apps
NDB API (C++)
Apps
PHP PERL
Apps

Schema-Free apps
 Rapid application evolution
– New types of data constantly
added
– No time to get schema
extended
– Missing skills to extend
schema
– Initially roll out to just a few
users
– Constantly adding to live
system

Cluster & Memcached – Schema-Free
<town:maidenhead,SL6>
key value
key value
Key Value
town:maidenhead SL6
generic table
Application view
SQL view

Cluster & Memcached - Configured Schema
prefix key value
key value
Prefix Table Key-col Val-col policy
town: map.zip town code cluste
r
Config tables
town ... code ...
maidenhead ... SL6 ...
map.zip
Application view
SQL view

MySQL 5.6 Memcached with InnoDB
0
10000
20000
30000
40000
50000
60000
70000
80000
8 32 128 512
TPS
Client Connections
MySQL 5.6: NoSQL
Benchmarking
Memcach
ed API
SQL
Clients and Applications
MySQL Server
Memcached Plug-in
innodb_
memcached
local cache
(optional)
Handler API InnoDB API
InnoDB Storage Engine
mysqld process
SQL Memcached Protocol
Up to 9x Higher “SET / INSERT” Throughput

MySQL Cluster 7.3: Node.js NoSQL API
 Native JavaScript access to MySQL Cluster
– End-to-End JavaScript: browser to the app &
DB
– Storing and retrieving JavaScript objects
directly in MySQL Cluster
– Eliminate SQL transformation
 Implemented as a module for node.js
– Integrates Cluster API library within the web app
 Couple high performance, distributed apps,
with high performance distributed database
 Optionally routes through MySQL Server
V8 JavaScript Engine
MySQL Cluster Node.js Module
Clients

MySQL Cluster NoSQL API for Node.js & FKs
 FKs enforced on all APIs:
{ message: 'Error',
sqlstate: '23000',
ndb_error: null,
cause:
{ message: 'Foreign key constraint violated: No parent row found
[255]',
sqlstate: '23000',
ndb_error:
{ message: 'Foreign key constraint violated: No parent row found',
code: 255,
classification: 'ConstraintViolation',
handler_error_code: 151,
status: 'PermanentError' },
cause: null } }

Next Steps
Learn More
• www.mysql.com/cluster
• Authentic MySQL Curriculum:
http://oracle.com/education/mysql
Try it Out
• dev.mysql.com/downloads/cluster/
Let us know what you think
• clusterdb.com
• @clusterdb
• forums.mysql.com/list.php?25

OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds

Similar to OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds (20)

More from Andrew Morgan

More from Andrew Morgan (13)

Recently uploaded

Recently uploaded (20)

OUG Scotland 2014 - NoSQL and MySQL - The best of both worlds