MongoDB World 2018: Breaking the Mold - Redesigning Dell's E-Commerce PlatformMongoDB
Similar to MongoDB 101 & Beyond: Get Started in MongoDB 3.0, Preview 3.2 & Demo of Ops Manager, Muthu Chinnasamy, Sr. Solutions Architect, MongoDB (20)
9. Pluggable Storage Engines
Content
Repo
IoT Sensor
Backend
Ad Service
Customer
Analytics
Archive
MongoDB Query Language (MQL) + Native Drivers
MongoDB Document Data Model
MMAP V1 WT In-Memory ? ?
Supported in MongoDB 3.0 Future Possible Storage Engines
Management
Security
Example Future State
Experimental
11. Additional flag when launching 3.0.x mongod:
--storageEngine=wiredTiger
How Do I run WiredTiger?
12. • Cannot copy raw data files. WT will fail to start
if it finds wrong data format
• Two ways to perform an engine upgrade:
• Mongodump/Mongorestore
• Add new replica member with WT
• You can run replicas with mixed storage
engines
How Do I Upgrade to it?
13. • Document-level concurrency
• Compression
• Consistency without journaling
• Better performance for write heavy
workloads
• Vertically scalable as it allows full hardware
utilization
• Index placement on faster storage devices
Why is WiredTiger Awesome?
14. mmapv1 2.6 concurrency model
Lock manager has a toll booth for each database
Reads and writes must acquire database level lock
15. mmapv1 3.0 concurrency model
Lock manager has a toll booth for each collection
Reads and writes must acquire collection level lock
16. 7-10x Better Write Performance
No toll booths!
Atomic update replaces latching/locking
Yields on write contention to same document
17. Up to 80% Compression
• WiredTiger uses snappy compression by default
in MongoDB
• Supported compression algorithms
– snappy [default]: good compression, low overhead
– zlib: better compression, more CPU
– none
• Indexes are compressed using prefix
compression
– allows compression in RAM
500GB 100GB
20. • WiredTiger cache stores uncompressed data
• Ideally, working set fits in WT cache
1. File system cache stores compressed data
2. Uses higher value of 50% of RAM or 1GB
WiredTiger Cache
--wiredTigerCacheSizeGB {arg}
23. How Ops Manager Helps You
Scale EasilyMeet SLAs
Best Practices,
Automated
Cut Management
Overhead
23
24. Zero Downtime Cluster Upgrades
Manual Steps
• Manual steps upgrading a 2.6.11 sharded cluster to
3.0.6 version, on enterprise edition, and convert the
storage engine to wiredTiger
24
Upgrades, downgrades
150+ steps
Example Deployment – 6 Servers
25. Zero Downtime Cluster Upgrades
Ops Manager Automation
• Using Ops Manager automation
• Start with a 2.6.11 Community cluster
• Upgrade to 3.0.6 Enterprise cluster
• Convert storage Engine to wiredTiger
Turn long and complicated manual processes into
single click operations with Ops Manager automation
Ensure availability during system upgrades
25
29. MongoDB 3.2 New Features
• Fast failovers
– 2-seconds election times
• Support for more than 3 data centers
– Config servers as replica sets
• Cluster read-committed
– readConcern: { level: <majority|local>
• WiredTiger default storage engine
• Aggregation framework enhancements
– New stages $sample
– New arithmetic operators & accumulators
• Mongo shell enhancements
31. MongoDB 3.2 – Left Outer Joins
• Create a new field in aggregated documents "cust_orders"
containing array of documents from "orders" collection
• Similar to where "customers"."_id"="orders"."cust_id"
Db.customers.aggregate([
{
$lookUp:
{ from: "orders",
localField: "_id",
foreignField: "cust_id",
as: "cust_orders"
}
}
])
32. MongoDB 3.2 – Encryption at rest
• Encryption is a database-level option
– Master key & local keys allow for easier key
management and key rotation for compliance
• Supports standard 3rd party key management
appliances (KMIP)
• Default encryption schema is AES256-CBC
– Also have GCM
• FIPS 140-2 compliant option
33. MongoDB 3.2 – Partial Indexes
> db.customers.createIndex({ state:1}, { partialFilterExpression: { active: {$eq : 'Y'}}})
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1
}
Index only for docs
matching the filter
expression