Your SlideShare is downloading. ×
MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference presentation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference presentation

5,295
views

Published on

MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference presentation

MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference presentation

Published in: Technology

0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,295
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
113
Comments
0
Likes
9
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Enteros, Inc. MongoDB Performance Tuning Load Testing NoSQL Now! 2013 Conference San Jose ron@enteros.com Ron Warshawsky August 23, 2013
  • 2. 2 MongoDB Overview Agenda Overview performance tuning Overview load testing
  • 3. 3 MongoDB Areas Of Performance Tuning OS tuning Storage tuning Database tuning Load Testing
  • 4. 4 MongoDB Areas Of Performance Tuning • OS Tuning • Follow ulimitsper mongo manual • http://docs.mongodb.org/manual/reference/ulimit/ • Follow production notes per mongo manual • http://docs.mongodb.org/manual/administration/production-notes/ • No atime on fs • No hugepages • Use NTT to sync time between the nodes • Try “tuned” • Best when used in staging/load testing environment under realisticload • CPU, Network and Storage tuning
  • 5. 5 MongoDB Areas Of Performance Tuning • Storage Tuning • RAID 10 • Ext4 or XFS • Delaysharding with better I/O • SSD + FlashCache – shutterflytested • https://github.com/facebook/flashcache/ • Provisioned IOPS on AWS
  • 6. 6 MongoDB Areas Of Performance Tuning • Storage Tuning • Ext4 + external journal on SSD – good I/O results • Make sure to test • journal_async_commit is used • http://www.raid6.com.au/posts/fs_ext4_external_journal/
  • 7. 7 MongoDB Areas Of Performance Tuning Softlayerstudy/ benchmarking “MongoDB Performance Analysis: Bare Metal v. Virtual” http://blog.softlayer.com/2012/mongodb-performance-analysis-bare-metal-v- virtual/ Highlights: • When a working data set is smaller than available memory, query performance increases. • The number of clients performing queries has an impact on query performance because more data is being actively cached at a rapid rate. • The addition of a separate Journal Mount volume significantly improves performance. Because the Small (SM) engineered server does not include a secondary mount for Journals, whenever MongoDB began to journal, the disk I/O associated with journaling was disruptive to the query and update operations performed on the Data Mount. • The best deployments in terms of operations per second,stability and control were the configurations with a RAID10 SSD Data Mount and a RAID1 SSD Journal Mount. These configurations are available in both Medium and Large offerings,and I’d highly recommend them.
  • 8. 8 MongoDB Areas Of Performance Tuning OS Performancemonitoring and analysis tools: • A) Linux utilities: iostat, vmstat, mpstat, sar, free -tm • B) Open source monitoring / performance analysissystems: Nagios, Cacti • http://tag1consulting.com/blog/mongodb-cacti-graphs • https://github.com/mzupan/nagios-plugin-mongodb C) Oracle OS Watcher– Free tool, performance collection and chart generator, easy to operate, very reliable, covers CPU, processes, top processes, I/O, network, etc OSWatcher Black Box Analyzer User Guide (Doc ID 461053.1)
  • 9. 9 MongoDB Areas Of Performance Tuning Database Tuning • Database configuration • Preallocatespace (noprealloc = false) • journalCommitInterval – range of 2 to 300, default 100. Can be adjusted to for better performance • Option to disable services: 1. HTTP Interface using nohttpinterface = true. 2. ScriptingEngine using noscripting = true (server-side java script). 3. RESTservice using rest = false.
  • 10. 10 MongoDB Areas Of Performance Tuning Database Tuning • Database monitoring tools • Mongostat– based on “eBay” “freecon” utility. • Use “–discover” option for membersof replicaSet / shared cluster • Displays critical performance metrics: • Flushes/Faults/locked db/idxmiss/qr/qw • Collect 24x7 and save for analysis
  • 11. 11 MongoDB Areas Of Performance Tuning Database Tuning • Database monitoring tools • Mongotop • Per collection I/O time tracking • Per database lock tracking (--lock option)
  • 12. 12 MongoDB Areas Of Performance Tuning Database networktraffic analysis • Database networkmonitoringtools • Mongosniff– sniffing MongoDb requests • Can capture invalid BSON requests from a network (--objcheck option) • Wireshark– support for MongoDB protocol (not BSON), great options for network performanceand latencyanalysis
  • 13. 13 MongoDB Areas Of Performance Tuning Database Tuning • Collections/ Fragmentation Collections in a MongoDB database can become fragmented. This can be a particularly serious problem if data usage patterns are relatively unstructured. In the long run, this will result in databases taking up more space on disk and in RAM to hold the same amount of data, it will make many database operations noticeably slower, and it can reduce overall query capacity significantly. compactcollection: db.runCommand ( { compact: '<collection>', paddingBytes: 100 } ) Automate compaction: http://blog.parse.com/2013/03/26/always-be-compacting/ ./mongo_compact.rb -d userdata1,userdata2,userdata3
  • 14. 14 MongoDB Areas Of Performance Tuning Database Tuning Profiler db.setProfilingLevel(level, slowms) db.setProfilingLevel(1)– collects slow requests Output: system.profile.op – operation system.profile.ns – namespace system.profile.ntoreturn – documentsreturned system.profile.nscanned – documentsscanned … http://docs.mongodb.org/manual/reference/database-profiler/ Profilerdeep dive https://speakerdeck.com/kgorman/mongodb-profiler-deep-dive
  • 15. 15 MongoDB Areas Of Performance Tuning Database Tuning • Indexes • Explain db.collection.find({…}).explain() Watch for scans • Indexes + Profiler= DEX - suggest what indexesare needed https://github.com/jwilder/mongodb-tools http://architects.dzone.com/articles/mongodb-performance-tuning- dex
  • 16. 16 MongoDB Areas Of Performance Tuning Database Monitoring and performance analysisUI a) MongoDB monitoring service (MMS) b) Open Source monitoringsoftware http://docs.mongodb.org/manual/administration/monitoring/ c) Enteros High Load Capture for MongoDB
  • 17. 17 MongoDB Areas Of Performance Tuning • Replication/ Replica set tuning Watch for replicationlag Problem: a) Weaksecondary b) Write bursts c) Index build d) Secondarylocked for backup e) Secondaryoffline Measure lag: db.printSlaveReplicationInfo() db.printReplicationInfo() Get time difference MMS provides replication lag chart and alerts
  • 18. 18 MongoDB Areas Of Performance Tuning • Sharding Tuning • Shard size • Migration of chunks • Failed migrationof chunks
  • 19. 19 MongoDB Load Testing Load testing 1) Whyload testing? • Validate upgrades • Validate fixes • Validate hardware and platform changes (to / from cloud, etc) • Validate multiplesof production load
  • 20. 20 MongoDB Load Testing 2) Options for Load testing A) BenchmarkLoad Testing • Mongo-Perf - Create standard TPCC-like load on MongoDB • https://github.com/mongodb/mongo-perf B) Disk I/O load testing • Mongoperf • Iozone http://www.iozone.org/
  • 21. 21 MongoDB Load Testing C) Real traffic Load Testing Parse PCAP files and generate production-like load http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire- protocol/ Create custom tool Enteros Load2Test (pre-GA)
  • 22. Understandbusiness requirements and it’s impact on production system design Designfor performance, scalability and availability Implement monitoring and performance analysis across layers Implement reliable production-like load testing MongoDB Summary
  • 23. Enteros, Inc http://www.enteros.com/ Enteros is an innovative software company specializing in Performance Management and Load Testing Software for Production Databases - RDBMS and NOSQL/Big Data Enteros solutions enable IT professionals to identify and remediate performance problems in business-critical databases with unprecedented speed, accuracy and scope. Ron Warshawsky; ron@enteros.com 408-207-8408 MongoDB Summary