MySQL Document Store
2017.01.14
Neowiz Games 강한글
1
MySQL Power Group
MySQL PowerGroup 2017년 1차 세미나
2
relational and document models
schema and schemaless
structured and unstructured
Combining SQL and NoSQL
MySQL Document Store
JSON
virtual columns
X DevAPI
CRUD
Transaction
Performance
X Protocol
X Plug-in
X Session
Node.js
MySQL Shell
MySQL Connector
MySQL PowerGroup 2017년 1차 세미나
3
relational and document models
schema and schemaless
structured and unstructured
“Combining SQL and NoSQL”
MySQL Document Store
JSON
virtual columns
X DevAPI
CRUD
Transaction
Performance
X Protocol
X Plug-in
X Session
Node.js
MySQL Shell
MySQL Connector
MySQL PowerGroup 2017년 1차 세미나
4
RDB vs Document Store Systems
for developers
MySQL Document Store - Combining SQL and NoSQL
Easier to get started
Upfront effort @ design stage
Simple deployment
CRUD
DevAPI NoSQL interface
RDB vs Document Store Systems
for DBAs
Performance
Indexing JSON data
Replication, backup, restore
Schema changes
Transaction
SQL
Data integrity
MySQL PowerGroup 2017년 1차 세미나
5
MySQL Document Store - Combining SQL and NoSQL
“지금보다 좋아지나요?”
“뭐가요?”
“얼마나요?”
MySQL PowerGroup 2017년 1차 세미나
6
TEXT
MySQL Document Store - Beginner
or varchar(4000)
or blob
MySQL PowerGroup 2017년 1차 세미나
7
“텍스트 컬럼 하나 만들어 주시면
알아서 쓸게요”
MySQL Document Store - Beginner
MySQL PowerGroup 2017년 1차 세미나
8
MySQL Document Store - Beginner
MySQL PowerGroup 2017년 1차 세미나
9
“툴에서 사용하기 번거로워요”
“데이터 수정하기 어려워요”
“개발자마다 형태가 달라요”
MySQL Document Store - Beginner
MySQL PowerGroup 2017년 1차 세미나
10
JSON
MySQL Document Store - Intermediate
MySQL PowerGroup 2017년 1차 세미나
11
구성
Native JSON datatype
JSON Functions
Generated Columns
특징
utf8mb4 character set
Optimized for read intensive workload
Parse and validation on insert
Fast access to array cells by index
스펙
Document Validation
Binary Format
CREATE TABLE `flags` (
`doc` json DEFAULT NULL,
`_id` varchar(32) GENERATED ALWAYS AS (json_unquote(json_extract(`doc`,'$._id'))) STORED NOT NULL,
PRIMARY KEY (`_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
;
MySQL Document Store - Intermediate
MySQL PowerGroup 2017년 1차 세미나
12
0
100
200
300
400
500
600
700
Insert select
text_no_ind
json_no_ind
text_ind
json_ind
0
5
10
15
20
25
30
35
select count(*)
text_no_ind
json_no_ind
text_ind
json_ind
MySQL PowerGroup 2017년 1차 세미나
13
0
50
100
150
200
250
300
select count(distinct)
text_no_ind
json_no_ind
text_ind
json_ind
0.13
3.66
7.63
11.32
15.82
19.35
22.47
25.34
29.15
32.5
0
50
100
150
200
250
300
select count(distinct) where id =
text_ind_where
json_ind_where
text_ind
json_ind
MySQL PowerGroup 2017년 1차 세미나
14
JSON_ARRAY_APPEND()
JSON_ARRAY_INSERT()
JSON_ARRAY()
JSON_CONTAINS_PATH()
JSON_CONTAINS()
JSON_DEPTH()
JSON_EXTRACT()
JSON_INSERT()
JSON_KEYS()
JSON_LENGTH()
explain select doc->'$.GNP', doc->'$.Name' from countryinfo where doc->'$._id' = 'SEA';
+----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| 1 | SIMPLE | countryinfo | NULL | const | PRIMARY | PRIMARY | 98 | const | 1 | 100.00 | NULL |
+----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
explain select doc->'$.GNP', doc->'$.Name' from countryinfo where _id = 'SEA';
+----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| 1 | SIMPLE | countryinfo | NULL | const | PRIMARY | PRIMARY | 98 | const | 1 | 100.00 | NULL |
+----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
select doc from countryinfo
where json_extract(doc,"$.GNP") > 500000;
select doc from countryinfo
where doc->'$.GNP' > 500000;
JSON_MERGE()
JSON_OBJECT()
JSON_QUOTE()
JSON_REMOVE()
JSON_REPLACE()
JSON_SEARCH()
JSON_SET()
JSON_TYPE()
JSON_UNQUOTE()
JSON_VALID()
MySQL Document Store - Intermediate
MySQL PowerGroup 2017년 1차 세미나
15
“쿼리 만들기 어려워요”
“좀 더…”
MySQL Document Store - Intermediate
MySQL PowerGroup 2017년 1차 세미나
16
const mysqlx = require('mysqlx');
mysqlx.getSession({
host: 'localhost',
dbUser: 'hank',
dbPassword: 'hank'
}).then(session => {
const collection = session.getSchema('world_x').getCollection('countryinfo');
return Promise.all([
collection.add(
{
_id: "SEA",
geography: {
Continent: "Europe",
Region: "British Islands",
SurfaceArea: 193
}
}).execute();
session.close();
])
}).catch(err => {
console.log(err);
});
const collection = session.getSchema('world_x').getCollection('countryinfo');
collection.find("GNP > 500000")
.sort(["IndepYear desc"])
.limit(10)
.execute(doc => console.log(doc))
.then(() => console.log("Success")
.catch((err) => console.log("Error", err);
var query = connection.query('select * from world_x.countryinfo where
GNP > 500000 order by IndepYear desc limit 10',function(err,rows){
console.log(rows);
res.json(rows);
});
X
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
17
db.countryinfo.find("_id = 'AUS'")
db.countryinfo.find("GNP > 500000 and demographics.Population
< 100000000")
db.countryinfo.find("Name = :country").bind("country", "Italy")
db.countryinfo.find("GNP > 5000000").fields(["GNP", "Name"])
MySQL Document Store - Advanced
SELECT doc FROM `world_x`.`countryinfo` WHERE (`_id` = 'AUS')
SELECT doc FROM `world_x`.`countryinfo`
WHERE ((JSON_EXTRACT(doc,'$.GNP') > 500000)
AND (JSON_EXTRACT(doc,'$.demographics.Population')
< 100000000))
set @country = 'Italy';
SELECT doc FROM `world_x`.`countryinfo` WHERE
(JSON_EXTRACT(doc,'$.Name') = @country)
SELECT JSON_OBJECT('GNP', JSON_EXTRACT(doc,'$.GNP'),'Name',
JSON_EXTRACT(doc,'$.Name')) AS doc
FROM `world_x`.`countryinfo`
WHERE (JSON_EXTRACT(doc,'$.GNP') > 5000000)
MySQL PowerGroup 2017년 1차 세미나
18
JSON + x
MySQL Document Store - Advanced
X DevAPI
X Session
X DevAPI Connector
X Protocol
X Plug-in
MySQL Shell
MySQL PowerGroup 2017년 1차 세미나
19
X DevAPI
X Session
X DevAPI Connector
X Protocol
X Plug-in
MySQL Shell
New, modern, async developer API for CRUD and
SQL operations on top of X Protocol
Introduces Collections as new Schema object
Focus on working with data via CRUD operations
Get away from traditional SQL
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
20
X DevAPI
X Session
X DevAPI Connector
X Protocol
X Plug-in
MySQL Shell
Connecton to an abstracted "service", for example:
Single instance MySQL server
MySQL InnoDB Cluster
Sharded MySQL InnoDB Cluster
Document Store API Only (SQL not allowed currently)
Transparent transaction routing
Future proof for scalability
mysqlsh –x / .getSession
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
21
X DevAPI
X Session / Node Session
X DevAPI Connector
X Protocol
X Plug-in
MySQL Shell
Connection to a specific MySQL server instance
Document Store and SQL operations allowed
Limited transaction routing
Unrestricted access
mysqlsh –node / .GetNodeSession
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
22
X DevAPI
X Session
X DevAPI Connector
X Protocol
X Plug-in
MySQL Shell
Support for X DevAPI and X Session
MySQL Shell 1.0.3
Connector/J 7.0
Connector/Net 7.0
Connector/Node.js1.0
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
23
X DevAPI
X Session
X DevAPI Connector
X Protocol
X Plug-in
MySQL Shell
New MySQL client protocol based on top of
industry standard
Works for both, CRUD and SQL operations
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
24
X DevAPI
X Session
X DevAPI Connector
X Protocol
X Plug-in
MySQL Shell
Plugin for MySQL server (since 5.7.12)
New protocol for MySQL with support for document
operations
Handles client connections on a separate TCP port
(default 33060 vs 3306)
Translate X Protocol document operations to SQL
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
25
X DevAPI
X Session
X DevAPI Connector
X Protocol
X Plug-in
MySQL Shell
New scriptable command line shell for MySQL Script
in Python and JavaScript (or traditional SQL)
DevAPI for Document Store
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
26
db.countryinfo.find()
select * from countryinfo;
MySQL Document Store - Advanced
MySQL PowerGroup 2017년 1차 세미나
27
Demonstration and preliminary use only
Enhanced Storage Engine for document store system
JSON native type (partial update / partial streaming / aggregate functions)
Generated columns
MySQL Document Store - in advance
MySQL PowerGroup 2017년 1차 세미나
28
Introduction to the MySQL Document Store
https://www.youtube.com/watch?v=1Dk517M-_7o
MySQL 5.7 and JSON; New Opportunities for Developers [CON8005]
Discover the MySQL Document Store [CON4662]
MySQL Document Store; A Deep Dive into Writing Scalable Applications [HOL6448]
MySQL X Protocol; Talking to MySQL Directly over the Wire [CON4231]
Using MySQL as a Document Store
http://dev.mysql.com/doc/refman/5.7/en/document-store.html
MySQL Document Store – References

MySQL Document Store

  • 1.
    MySQL Document Store 2017.01.14 NeowizGames 강한글 1 MySQL Power Group
  • 2.
    MySQL PowerGroup 2017년1차 세미나 2 relational and document models schema and schemaless structured and unstructured Combining SQL and NoSQL MySQL Document Store JSON virtual columns X DevAPI CRUD Transaction Performance X Protocol X Plug-in X Session Node.js MySQL Shell MySQL Connector
  • 3.
    MySQL PowerGroup 2017년1차 세미나 3 relational and document models schema and schemaless structured and unstructured “Combining SQL and NoSQL” MySQL Document Store JSON virtual columns X DevAPI CRUD Transaction Performance X Protocol X Plug-in X Session Node.js MySQL Shell MySQL Connector
  • 4.
    MySQL PowerGroup 2017년1차 세미나 4 RDB vs Document Store Systems for developers MySQL Document Store - Combining SQL and NoSQL Easier to get started Upfront effort @ design stage Simple deployment CRUD DevAPI NoSQL interface RDB vs Document Store Systems for DBAs Performance Indexing JSON data Replication, backup, restore Schema changes Transaction SQL Data integrity
  • 5.
    MySQL PowerGroup 2017년1차 세미나 5 MySQL Document Store - Combining SQL and NoSQL “지금보다 좋아지나요?” “뭐가요?” “얼마나요?”
  • 6.
    MySQL PowerGroup 2017년1차 세미나 6 TEXT MySQL Document Store - Beginner or varchar(4000) or blob
  • 7.
    MySQL PowerGroup 2017년1차 세미나 7 “텍스트 컬럼 하나 만들어 주시면 알아서 쓸게요” MySQL Document Store - Beginner
  • 8.
    MySQL PowerGroup 2017년1차 세미나 8 MySQL Document Store - Beginner
  • 9.
    MySQL PowerGroup 2017년1차 세미나 9 “툴에서 사용하기 번거로워요” “데이터 수정하기 어려워요” “개발자마다 형태가 달라요” MySQL Document Store - Beginner
  • 10.
    MySQL PowerGroup 2017년1차 세미나 10 JSON MySQL Document Store - Intermediate
  • 11.
    MySQL PowerGroup 2017년1차 세미나 11 구성 Native JSON datatype JSON Functions Generated Columns 특징 utf8mb4 character set Optimized for read intensive workload Parse and validation on insert Fast access to array cells by index 스펙 Document Validation Binary Format CREATE TABLE `flags` ( `doc` json DEFAULT NULL, `_id` varchar(32) GENERATED ALWAYS AS (json_unquote(json_extract(`doc`,'$._id'))) STORED NOT NULL, PRIMARY KEY (`_id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 ; MySQL Document Store - Intermediate
  • 12.
    MySQL PowerGroup 2017년1차 세미나 12 0 100 200 300 400 500 600 700 Insert select text_no_ind json_no_ind text_ind json_ind 0 5 10 15 20 25 30 35 select count(*) text_no_ind json_no_ind text_ind json_ind
  • 13.
    MySQL PowerGroup 2017년1차 세미나 13 0 50 100 150 200 250 300 select count(distinct) text_no_ind json_no_ind text_ind json_ind 0.13 3.66 7.63 11.32 15.82 19.35 22.47 25.34 29.15 32.5 0 50 100 150 200 250 300 select count(distinct) where id = text_ind_where json_ind_where text_ind json_ind
  • 14.
    MySQL PowerGroup 2017년1차 세미나 14 JSON_ARRAY_APPEND() JSON_ARRAY_INSERT() JSON_ARRAY() JSON_CONTAINS_PATH() JSON_CONTAINS() JSON_DEPTH() JSON_EXTRACT() JSON_INSERT() JSON_KEYS() JSON_LENGTH() explain select doc->'$.GNP', doc->'$.Name' from countryinfo where doc->'$._id' = 'SEA'; +----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+ | 1 | SIMPLE | countryinfo | NULL | const | PRIMARY | PRIMARY | 98 | const | 1 | 100.00 | NULL | +----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+ explain select doc->'$.GNP', doc->'$.Name' from countryinfo where _id = 'SEA'; +----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+ | 1 | SIMPLE | countryinfo | NULL | const | PRIMARY | PRIMARY | 98 | const | 1 | 100.00 | NULL | +----+-------------+-------------+------------+-------+---------------+---------+---------+-------+------+----------+-------+ select doc from countryinfo where json_extract(doc,"$.GNP") > 500000; select doc from countryinfo where doc->'$.GNP' > 500000; JSON_MERGE() JSON_OBJECT() JSON_QUOTE() JSON_REMOVE() JSON_REPLACE() JSON_SEARCH() JSON_SET() JSON_TYPE() JSON_UNQUOTE() JSON_VALID() MySQL Document Store - Intermediate
  • 15.
    MySQL PowerGroup 2017년1차 세미나 15 “쿼리 만들기 어려워요” “좀 더…” MySQL Document Store - Intermediate
  • 16.
    MySQL PowerGroup 2017년1차 세미나 16 const mysqlx = require('mysqlx'); mysqlx.getSession({ host: 'localhost', dbUser: 'hank', dbPassword: 'hank' }).then(session => { const collection = session.getSchema('world_x').getCollection('countryinfo'); return Promise.all([ collection.add( { _id: "SEA", geography: { Continent: "Europe", Region: "British Islands", SurfaceArea: 193 } }).execute(); session.close(); ]) }).catch(err => { console.log(err); }); const collection = session.getSchema('world_x').getCollection('countryinfo'); collection.find("GNP > 500000") .sort(["IndepYear desc"]) .limit(10) .execute(doc => console.log(doc)) .then(() => console.log("Success") .catch((err) => console.log("Error", err); var query = connection.query('select * from world_x.countryinfo where GNP > 500000 order by IndepYear desc limit 10',function(err,rows){ console.log(rows); res.json(rows); }); X MySQL Document Store - Advanced
  • 17.
    MySQL PowerGroup 2017년1차 세미나 17 db.countryinfo.find("_id = 'AUS'") db.countryinfo.find("GNP > 500000 and demographics.Population < 100000000") db.countryinfo.find("Name = :country").bind("country", "Italy") db.countryinfo.find("GNP > 5000000").fields(["GNP", "Name"]) MySQL Document Store - Advanced SELECT doc FROM `world_x`.`countryinfo` WHERE (`_id` = 'AUS') SELECT doc FROM `world_x`.`countryinfo` WHERE ((JSON_EXTRACT(doc,'$.GNP') > 500000) AND (JSON_EXTRACT(doc,'$.demographics.Population') < 100000000)) set @country = 'Italy'; SELECT doc FROM `world_x`.`countryinfo` WHERE (JSON_EXTRACT(doc,'$.Name') = @country) SELECT JSON_OBJECT('GNP', JSON_EXTRACT(doc,'$.GNP'),'Name', JSON_EXTRACT(doc,'$.Name')) AS doc FROM `world_x`.`countryinfo` WHERE (JSON_EXTRACT(doc,'$.GNP') > 5000000)
  • 18.
    MySQL PowerGroup 2017년1차 세미나 18 JSON + x MySQL Document Store - Advanced X DevAPI X Session X DevAPI Connector X Protocol X Plug-in MySQL Shell
  • 19.
    MySQL PowerGroup 2017년1차 세미나 19 X DevAPI X Session X DevAPI Connector X Protocol X Plug-in MySQL Shell New, modern, async developer API for CRUD and SQL operations on top of X Protocol Introduces Collections as new Schema object Focus on working with data via CRUD operations Get away from traditional SQL MySQL Document Store - Advanced
  • 20.
    MySQL PowerGroup 2017년1차 세미나 20 X DevAPI X Session X DevAPI Connector X Protocol X Plug-in MySQL Shell Connecton to an abstracted "service", for example: Single instance MySQL server MySQL InnoDB Cluster Sharded MySQL InnoDB Cluster Document Store API Only (SQL not allowed currently) Transparent transaction routing Future proof for scalability mysqlsh –x / .getSession MySQL Document Store - Advanced
  • 21.
    MySQL PowerGroup 2017년1차 세미나 21 X DevAPI X Session / Node Session X DevAPI Connector X Protocol X Plug-in MySQL Shell Connection to a specific MySQL server instance Document Store and SQL operations allowed Limited transaction routing Unrestricted access mysqlsh –node / .GetNodeSession MySQL Document Store - Advanced
  • 22.
    MySQL PowerGroup 2017년1차 세미나 22 X DevAPI X Session X DevAPI Connector X Protocol X Plug-in MySQL Shell Support for X DevAPI and X Session MySQL Shell 1.0.3 Connector/J 7.0 Connector/Net 7.0 Connector/Node.js1.0 MySQL Document Store - Advanced
  • 23.
    MySQL PowerGroup 2017년1차 세미나 23 X DevAPI X Session X DevAPI Connector X Protocol X Plug-in MySQL Shell New MySQL client protocol based on top of industry standard Works for both, CRUD and SQL operations MySQL Document Store - Advanced
  • 24.
    MySQL PowerGroup 2017년1차 세미나 24 X DevAPI X Session X DevAPI Connector X Protocol X Plug-in MySQL Shell Plugin for MySQL server (since 5.7.12) New protocol for MySQL with support for document operations Handles client connections on a separate TCP port (default 33060 vs 3306) Translate X Protocol document operations to SQL MySQL Document Store - Advanced
  • 25.
    MySQL PowerGroup 2017년1차 세미나 25 X DevAPI X Session X DevAPI Connector X Protocol X Plug-in MySQL Shell New scriptable command line shell for MySQL Script in Python and JavaScript (or traditional SQL) DevAPI for Document Store MySQL Document Store - Advanced
  • 26.
    MySQL PowerGroup 2017년1차 세미나 26 db.countryinfo.find() select * from countryinfo; MySQL Document Store - Advanced
  • 27.
    MySQL PowerGroup 2017년1차 세미나 27 Demonstration and preliminary use only Enhanced Storage Engine for document store system JSON native type (partial update / partial streaming / aggregate functions) Generated columns MySQL Document Store - in advance
  • 28.
    MySQL PowerGroup 2017년1차 세미나 28 Introduction to the MySQL Document Store https://www.youtube.com/watch?v=1Dk517M-_7o MySQL 5.7 and JSON; New Opportunities for Developers [CON8005] Discover the MySQL Document Store [CON4662] MySQL Document Store; A Deep Dive into Writing Scalable Applications [HOL6448] MySQL X Protocol; Talking to MySQL Directly over the Wire [CON4231] Using MySQL as a Document Store http://dev.mysql.com/doc/refman/5.7/en/document-store.html MySQL Document Store – References