Enteros, Inc.
MongoDB
Performance Tuning
Load Testing
NoSQL Now! 2013 Conference
San Jose
ron@enteros.com
Ron Warshawsky
A...
2
MongoDB Overview
Agenda
Overview performance
tuning
Overview load testing
3
MongoDB Areas Of Performance Tuning
OS tuning
Storage tuning
Database tuning
Load Testing
4
MongoDB Areas Of Performance Tuning
• OS Tuning
• Follow ulimitsper mongo manual
• http://docs.mongodb.org/manual/refere...
5
MongoDB Areas Of Performance Tuning
• Storage Tuning
• RAID 10
• Ext4 or XFS
• Delaysharding with better I/O
• SSD + Fla...
6
MongoDB Areas Of Performance Tuning
• Storage Tuning
• Ext4 + external journal on SSD – good I/O results
• Make sure to ...
7
MongoDB Areas Of Performance Tuning
Softlayerstudy/ benchmarking
“MongoDB Performance Analysis: Bare Metal v. Virtual”
h...
8
MongoDB Areas Of Performance Tuning
OS Performancemonitoring and analysis tools:
• A) Linux utilities: iostat, vmstat, m...
9
MongoDB Areas Of Performance Tuning
Database Tuning
• Database configuration
• Preallocatespace (noprealloc = false)
• j...
10
MongoDB Areas Of Performance Tuning
Database Tuning
• Database monitoring tools
• Mongostat– based on “eBay” “freecon” ...
11
MongoDB Areas Of Performance Tuning
Database Tuning
• Database monitoring tools
• Mongotop
• Per collection I/O time tr...
12
MongoDB Areas Of Performance Tuning
Database networktraffic analysis
• Database networkmonitoringtools
• Mongosniff– sn...
13
MongoDB Areas Of Performance Tuning
Database Tuning
• Collections/ Fragmentation
Collections in a MongoDB database can ...
14
MongoDB Areas Of Performance Tuning
Database Tuning
Profiler
db.setProfilingLevel(level, slowms)
db.setProfilingLevel(1...
15
MongoDB Areas Of Performance Tuning
Database Tuning
• Indexes
• Explain
db.collection.find({…}).explain()
Watch for sca...
16
MongoDB Areas Of Performance Tuning
Database Monitoring and performance analysisUI
a) MongoDB monitoring service (MMS)
...
17
MongoDB Areas Of Performance Tuning
• Replication/ Replica set tuning
Watch for replicationlag
Problem:
a) Weaksecondar...
18
MongoDB Areas Of Performance Tuning
• Sharding Tuning
• Shard size
• Migration of chunks
• Failed migrationof chunks
19
MongoDB Load Testing
Load testing
1) Whyload testing?
• Validate upgrades
• Validate fixes
• Validate hardware and plat...
20
MongoDB Load Testing
2) Options for Load testing
A) BenchmarkLoad Testing
• Mongo-Perf - Create standard TPCC-like load...
21
MongoDB Load Testing
C) Real traffic Load Testing
Parse PCAP files and generate production-like load
http://docs.mongod...
Understandbusiness requirements
and it’s impact on production system
design
Designfor performance, scalability
and avail...
Enteros, Inc
http://www.enteros.com/
Enteros is an innovative software company specializing in
Performance Management and ...
Upcoming SlideShare
Loading in...5
×

MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference presentation

6,945

Published on

MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference presentation

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,945
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
156
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference presentation

  1. 1. Enteros, Inc. MongoDB Performance Tuning Load Testing NoSQL Now! 2013 Conference San Jose ron@enteros.com Ron Warshawsky August 23, 2013
  2. 2. 2 MongoDB Overview Agenda Overview performance tuning Overview load testing
  3. 3. 3 MongoDB Areas Of Performance Tuning OS tuning Storage tuning Database tuning Load Testing
  4. 4. 4 MongoDB Areas Of Performance Tuning • OS Tuning • Follow ulimitsper mongo manual • http://docs.mongodb.org/manual/reference/ulimit/ • Follow production notes per mongo manual • http://docs.mongodb.org/manual/administration/production-notes/ • No atime on fs • No hugepages • Use NTT to sync time between the nodes • Try “tuned” • Best when used in staging/load testing environment under realisticload • CPU, Network and Storage tuning
  5. 5. 5 MongoDB Areas Of Performance Tuning • Storage Tuning • RAID 10 • Ext4 or XFS • Delaysharding with better I/O • SSD + FlashCache – shutterflytested • https://github.com/facebook/flashcache/ • Provisioned IOPS on AWS
  6. 6. 6 MongoDB Areas Of Performance Tuning • Storage Tuning • Ext4 + external journal on SSD – good I/O results • Make sure to test • journal_async_commit is used • http://www.raid6.com.au/posts/fs_ext4_external_journal/
  7. 7. 7 MongoDB Areas Of Performance Tuning Softlayerstudy/ benchmarking “MongoDB Performance Analysis: Bare Metal v. Virtual” http://blog.softlayer.com/2012/mongodb-performance-analysis-bare-metal-v- virtual/ Highlights: • When a working data set is smaller than available memory, query performance increases. • The number of clients performing queries has an impact on query performance because more data is being actively cached at a rapid rate. • The addition of a separate Journal Mount volume significantly improves performance. Because the Small (SM) engineered server does not include a secondary mount for Journals, whenever MongoDB began to journal, the disk I/O associated with journaling was disruptive to the query and update operations performed on the Data Mount. • The best deployments in terms of operations per second,stability and control were the configurations with a RAID10 SSD Data Mount and a RAID1 SSD Journal Mount. These configurations are available in both Medium and Large offerings,and I’d highly recommend them.
  8. 8. 8 MongoDB Areas Of Performance Tuning OS Performancemonitoring and analysis tools: • A) Linux utilities: iostat, vmstat, mpstat, sar, free -tm • B) Open source monitoring / performance analysissystems: Nagios, Cacti • http://tag1consulting.com/blog/mongodb-cacti-graphs • https://github.com/mzupan/nagios-plugin-mongodb C) Oracle OS Watcher– Free tool, performance collection and chart generator, easy to operate, very reliable, covers CPU, processes, top processes, I/O, network, etc OSWatcher Black Box Analyzer User Guide (Doc ID 461053.1)
  9. 9. 9 MongoDB Areas Of Performance Tuning Database Tuning • Database configuration • Preallocatespace (noprealloc = false) • journalCommitInterval – range of 2 to 300, default 100. Can be adjusted to for better performance • Option to disable services: 1. HTTP Interface using nohttpinterface = true. 2. ScriptingEngine using noscripting = true (server-side java script). 3. RESTservice using rest = false.
  10. 10. 10 MongoDB Areas Of Performance Tuning Database Tuning • Database monitoring tools • Mongostat– based on “eBay” “freecon” utility. • Use “–discover” option for membersof replicaSet / shared cluster • Displays critical performance metrics: • Flushes/Faults/locked db/idxmiss/qr/qw • Collect 24x7 and save for analysis
  11. 11. 11 MongoDB Areas Of Performance Tuning Database Tuning • Database monitoring tools • Mongotop • Per collection I/O time tracking • Per database lock tracking (--lock option)
  12. 12. 12 MongoDB Areas Of Performance Tuning Database networktraffic analysis • Database networkmonitoringtools • Mongosniff– sniffing MongoDb requests • Can capture invalid BSON requests from a network (--objcheck option) • Wireshark– support for MongoDB protocol (not BSON), great options for network performanceand latencyanalysis
  13. 13. 13 MongoDB Areas Of Performance Tuning Database Tuning • Collections/ Fragmentation Collections in a MongoDB database can become fragmented. This can be a particularly serious problem if data usage patterns are relatively unstructured. In the long run, this will result in databases taking up more space on disk and in RAM to hold the same amount of data, it will make many database operations noticeably slower, and it can reduce overall query capacity significantly. compactcollection: db.runCommand ( { compact: '<collection>', paddingBytes: 100 } ) Automate compaction: http://blog.parse.com/2013/03/26/always-be-compacting/ ./mongo_compact.rb -d userdata1,userdata2,userdata3
  14. 14. 14 MongoDB Areas Of Performance Tuning Database Tuning Profiler db.setProfilingLevel(level, slowms) db.setProfilingLevel(1)– collects slow requests Output: system.profile.op – operation system.profile.ns – namespace system.profile.ntoreturn – documentsreturned system.profile.nscanned – documentsscanned … http://docs.mongodb.org/manual/reference/database-profiler/ Profilerdeep dive https://speakerdeck.com/kgorman/mongodb-profiler-deep-dive
  15. 15. 15 MongoDB Areas Of Performance Tuning Database Tuning • Indexes • Explain db.collection.find({…}).explain() Watch for scans • Indexes + Profiler= DEX - suggest what indexesare needed https://github.com/jwilder/mongodb-tools http://architects.dzone.com/articles/mongodb-performance-tuning- dex
  16. 16. 16 MongoDB Areas Of Performance Tuning Database Monitoring and performance analysisUI a) MongoDB monitoring service (MMS) b) Open Source monitoringsoftware http://docs.mongodb.org/manual/administration/monitoring/ c) Enteros High Load Capture for MongoDB
  17. 17. 17 MongoDB Areas Of Performance Tuning • Replication/ Replica set tuning Watch for replicationlag Problem: a) Weaksecondary b) Write bursts c) Index build d) Secondarylocked for backup e) Secondaryoffline Measure lag: db.printSlaveReplicationInfo() db.printReplicationInfo() Get time difference MMS provides replication lag chart and alerts
  18. 18. 18 MongoDB Areas Of Performance Tuning • Sharding Tuning • Shard size • Migration of chunks • Failed migrationof chunks
  19. 19. 19 MongoDB Load Testing Load testing 1) Whyload testing? • Validate upgrades • Validate fixes • Validate hardware and platform changes (to / from cloud, etc) • Validate multiplesof production load
  20. 20. 20 MongoDB Load Testing 2) Options for Load testing A) BenchmarkLoad Testing • Mongo-Perf - Create standard TPCC-like load on MongoDB • https://github.com/mongodb/mongo-perf B) Disk I/O load testing • Mongoperf • Iozone http://www.iozone.org/
  21. 21. 21 MongoDB Load Testing C) Real traffic Load Testing Parse PCAP files and generate production-like load http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire- protocol/ Create custom tool Enteros Load2Test (pre-GA)
  22. 22. Understandbusiness requirements and it’s impact on production system design Designfor performance, scalability and availability Implement monitoring and performance analysis across layers Implement reliable production-like load testing MongoDB Summary
  23. 23. Enteros, Inc http://www.enteros.com/ Enteros is an innovative software company specializing in Performance Management and Load Testing Software for Production Databases - RDBMS and NOSQL/Big Data Enteros solutions enable IT professionals to identify and remediate performance problems in business-critical databases with unprecedented speed, accuracy and scope. Ron Warshawsky; ron@enteros.com 408-207-8408 MongoDB Summary
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×