Working with MongoDB as MySQL DBA. Comparing commands from MongoDB to MySQL, similarities and differences. Exploring replication features, failover and recovery, adjusting the variables and checking status and using DML, DDL with different storage engines

1. Working with MongoDB as
MySQL DBA
Date: Oct-5-2016

2. About us
Lead Database Consultant @Pythian
OSDB managed services since 2014
https://mk.linkedin.com/in/igorle
@igorLE
Lead Database Consultant
at @Pythian since 2015.
https://www.linkedin.com/in/martinarrieta
@MartinArrietaC

3. Overview
• What is MySQL, what MongoDB
• Replication in MongoDB compared to MySQL
• Resyncing nodes after crash
• Changing variables and configuration files
• Status check and processlist
• Security and authentication
• Schema design, DDL, DML
• Storage engine and concurrency
• Error and Slow query logs
• Backups and recovery

4. What is MySQL
• Open-source relational database management system
(RDBMS)
• Stores data in tables and uses Structured Query
Language (SQL) for database access
• Predefined database schema based requirements and
rules to govern the relationships between fields in your
tables
• Related information may be stored in separate tables,
but associated through the use of joins
• Data duplication is minimized

5. What is MongoDB
• Open-source database, stores data in JSON-like documents
that can vary in structure
• Related information is stored together for fast query access
through the MongoDB query language
• Uses dynamic schemas, creating records without first
defining the structure
• Changing the structure simply by adding new fields or
deleting existing ones
• Ability to represent hierarchical relationships, to store arrays,
and other more complex structures easily
• Documents in a collection need not have an identical set of
fields and denormalization of data is common

6. MySQL replication
• Group of mysqld processes that maintain the same data set
• Write operations go to the Master node, reads can go to
Slave
• All changes are recorded into binary log on the master
• Asynchronous replication, Semi-Synchronous replication,
virtually synchronous replication (Galera)
• Slave node replicates the events by copying binary logs as
relay logs
– IO Thread, SQL Thread
– Binary Logs format - Statement, Row, Mixed
• Expire logs days dynamic
• Replication filters

7. MySQL replication
MySQL Master
MySQL Process
Binary LogsMySQL Data
MySQL Slave
MySQL Process
SQL Thread
Relay logs
IO Thread MySQL Data

8. MongoDB Replication
• Group of mongod processes that maintain the same
data set
• Write operations go to the Primary node
• All changes are recorded on all nodes into operations
log - oplog
• Asynchronous replication, Statement based
• Secondaries replicate the Primary oplog
• Secondary can use sync source other Secondary
• Automatic failover
• Oplog size change
• Adding new node

9. MongoDB Replication

10. Crash recovery
MySQL
• InnoDB does crash recovery
• MyISAM can repair tables
• Restore from backup - reseed replication
• Does not handle the quorum
MongoDB
• Primary node crash. Election of new Primary from majority
• Replay events from Oplog - idempotent
• Clear data dir and restart node. Initial Sync
• Repair database

11. Failover
MySQL
• There is no build-in automatic failover
– Alternatives can be MHA, HAProxy, ProxySQL, etc.
• No option for automated initial sync
MongoDB
• Primary nodes goes down for more than 10 seconds
– "heartbeatTimeoutSecs" : 10
• Secondary node holds election
• Secondary with majority of votes becomes new Primary

12. Variables and configuration file
MongoDB
/etc/mongod.conf
systemLog:
destination: file
path: /var/log/mongodb/mongod.log
logAppend: true
storage:
dbPath: /mongodb/data
journal:
enabled: true
wiredTiger:
engineConfig:
cacheSizeGB: 2
net:
bindIp: 127.0.0.1
port: 27017
operationProfiling:
slowOpThresholdMs: 100
mode: off
MySQL
/etc/my.cnf
[client]
port = 3306
socket = /var/run/mysqld/mysql.sock
[mysqld]
port = 3306
user = mysql
socket = /var/run/mysqld/mysql.sock
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
log_error = hostname_error.log
slow_query_log_file = hostname_slow.log
innodb_buffer_pool_size = 128M
innodb_file_per_table = 1
[mysqldump]
max_allowed_packet = 64M

13. Server status
MongoDB
PRIMARY> db.serverStatus()
{ "host" : "mongodb1",
"version" : "3.0.11",
"process" : "mongod",
"pid" : NumberLong(15805),
"uptime" : 9522445,
…...
"uptimeEstimate" : 9409413,
"localTime" :
ISODate("2016-09-29T14:39:43.769Z",
…...
"connections" : {
"current" : 802,
"available" : 50398,
"totalCreated" :
NumberLong(644684)
}
MySQL
mysql> SHOW GLOBAL STATUS;
+--------------------------+------------+
| Variable_name | Value |
+--------------------------+------------+
| Aborted_clients | 0 |
| Aborted_connects | 0 |
| Connections | 30023 |
| Created_tmp_disk_tables | 0 |
| Created_tmp_tables | 8340 |
| Created_tmp_files | 60 |
...
| Threads_connected | 1 |
| Threads_running | 1 |
| Uptime | 80380 |
+--------------------------+------------+

14. Processlist
MongoDB
PRIMARY> db.currentOp()
{ "inprog" : [ {
"desc" : "conn531228",
"threadId" : "0xe856680",
"connectionId" : 531228,
"opid" : 1798444507,
"active" : true,
"secs_running" : 0,
"microsecs_running" : NumberLong(995),
"op" : "query",
"ns" : "hbrg.user.cart",
"query" : { "$query" : { "user" :
"C2ABBA1F-3555-3649-85D8-7A544F15EA94" }},
"planSummary" : "IXSCAN { user: 1 }",
......
} ] }
MySQL
mysql> SHOW FULL PROCESSLISTG
************* 1. row ***************
Id: 3123
User: igor
Host: localhost
db: test
Command: Query
Time: 0
State: NULL
Info: SHOW FULL PROCESSLIST

15. Security and authentication
MongoDB
security:
keyFile: <path-to-keyfile>
use admin
db.createUser( {
user: "admin",
pwd: "changeme1",
roles: [ { role:
"userAdminAnyDatabase", db: "admin" }
]
})
db.getSiblingDB("admin").auth("admin"
, "changeme1" )
MySQL
CREATE USER 'admin'@'localhost'
IDENTIFIED BY 'changeme1';
GRANT ALL ON app_db.*
TO 'admin' @'localhost';
There are not “Roles” in MySQL :(
MySQL 8 will support roles.
CREATE ROLE 'app_developer';
GRANT ALL ON app_db.* TO 'app_developer';
GRANT 'app_developer' TO
'dev1'@'localhost';

16. Security and authentication
MongoDB
Encrypt data in transit with SSL
/etc/mongod.conf
net:
ssl:
mode: requireSSL
PEMKeyFile:
/etc/ssl/mongodb.pem
CAFile: /etc/ssl/ca.pem
shell> mongo --ssl --host
hostname.example.com --sslCAFile
/etc/ssl/ca.pem
MySQL
Encrypt data in transit with SSL
/etc/my.cnf
[mysqld]
ssl-ca=ca.pem
ssl-cert=server-cert.pem
ssl-key=server-key.pem
mysql> GRANT USAGE ON *.*
TO user@'53.40.20.15' REQUIRE SSL;
shell> mysql --ssl-ca=ca.pem
--ssl-cert=client-cert.pem
--ssl-key=client-key.pem

17. Schema design, DDL, DML
MongoDB
PRIMARY> use test
switched to db test
PRIMARY> db.createCollection(
"users",
{ validator: { $and:
[ { name: { $type: "string" } },
{ status: { $in: [ "A", "P" ] } }
]
}
}
)
* Available from MongoDB 3.2
MySQL
mysql> CREATE DATABASE test;
mysql> USE test;
CREATE TABLE `users` (
`id` INT(10) NOT NULL AUTO_INCREMENT,
`age` CHAR(3) NULL DEFAULT NULL,
`status` ENUM('A','P') NOT NULL
DEFAULT 'P',
`name` VARCHAR(50) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;

18. Schema design, DDL, DML
MongoDB
PRIMARY> show collections
users
PRIMARY> db.users.insert({"age":
"45", "status": 'A', "name": "John"})
WriteResult({ "nInserted" : 1 })
PRIMARY> db.users.find().pretty()
{ "_id" :
ObjectId("57c5d9ab0b044cb923fffaf9")
,
"age" : "45",
"status" : "A",
"name" : "John"
}
MySQL
mysql> SHOW TABLES;
+-----------------+
| Tables_in_test |
+-----------------+
| users |
+-----------------+
mysql> INSERT INTO `users` (`age`,
`status`, `name`) VALUES ( '45', 'A',
'John');
mysql> SELECT * FROM users;
+----+------+--------+------+
| id | age | status | name |
+----+------+--------+------+
| 1 | 45 | A | John |
+----+------+--------+------+

19. Schema design, DDL, DML
MongoDB
PRIMARY> db.users.insert({"age":
"25", "status": 'R', "name": "Tom"})
WriteResult({ "nInserted" : 0,
"writeError" : {
"code" : 121,
"errmsg" :
"Document failed validation"
}})
PRIMARY> db.users.find().pretty()
{ "_id" :
ObjectId("57c5dbd67fd59bca3fb314ac")
,
"age" : "45",
"status" : "A",
"name" : "John"
}
MySQL
mysql> INSERT INTO `users` (`age`,
`status`, `name`) VALUES ( '25', 'R',
'Tom');
ERROR 1265 (01000): Data truncated
for column 'status' at row 1
mysql> SELECT * FROM users;
+----+------+--------+------+
| id | age | status | name |
+----+------+--------+------+
| 1 | 45 | A | John |
+----+------+--------+------+

20. Schema design, DDL, DML
MongoDB
PRIMARY> db.users.update({ "_id" :
ObjectId("57c5dbd67fd59bca3fb314ac")}
, {$set : {"status":'P'} } )
WriteResult({ "nMatched" : 1,
"nUpserted" : 0, "nModified" : 1 })
PRIMARY> db.users.remove( {} )
PRIMARY> ALTER collection users … ?
PRIMARY> db.users.update({ "_id" :
ObjectId("57c5dbd67fd59bca3fb314ac")}
, {$set : {"position":"DBA"} } )
MySQL
mysql> UPDATE users SET status='P'
WHERE id=1;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1
Warnings: 0
mysql> truncate users; delete from
users;
mysql> ALTER TABLE `users` ADD
COLUMN `position` VARCHAR(50) NULL
AFTER `name`;
mysql> UPDATE `users` SET
`position`='DBA' WHERE `id`=1;

21. Schema design, DDL, DML
MongoDB
PRIMARY> db.users.createIndex({"name":1})
PRIMARY>
db.users.find({"name":"John"}).explain()
{ ........
"winningPlan" : {
"stage" : "FETCH",
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : { "name" : 1 },
"indexName" : "name_1",
........
"indexBounds" : {"name" : [
"["John", "John"]"
] }
}
PRIMARY> db.users.drop( )
MySQL
mysql> ALTER TABLE `users` ADD INDEX
`name_idx` (`name`);
mysql> explain select * from users where
name = 'John'G
*********** 1. row **************
id: 1
select_type: SIMPLE
table: users
type: ref
possible_keys: name_idx
key: name_idx
……..
rows: 1
Extra: Using index condition
1 row in set (0.00 sec)
mysql> DROP TABLE users;

22. Storage engines
MongoDB
MMAPv1
WiredTiger
RocksDB
In-memory
PerconaFT
MySQL
MyISAM
InnoDB
MyRocks
Memory
CSV
Archive
Federated
Blackhole

23. Log files
MongoDB
Error log
Slow query log
/var/log/mongodb/mongod.log
MySQL
Error log
Slow query log
/var/log/mysql/error.log
/var/log/mysql/slow_query.log

24. Backups and recovery
MongoDB
• mongodump
– Logical backup
– Slow to create and restore
• File system snapshot
– LVM, EBS snapshot
• cp or rsync
– Copy files from freezed node
• Cloud manager
– Monthly subscription
– Point in time backup
• Ops manager (Enterprise)
– Yearly subscription
– Point in time backup
MySQL
• mysqldump
– Logical backup
– Slow to create and worse for
restore
• mydumper
– Logical backup
– Still slow but faster than
mysqldump
• xtrabackup
– Binary backup
– Fast to create and restore
• mysqlbackup
– NOT Open Source
– Binary backup

25. Questions?

26. We’re Hiring!
https://pythian.com