Download to read offline
Uploaded byDrew Stephens
Big Data DC - BenchPress
Palomino Labs, Inc. offers BenchPress, a high-performance benchmarking tool for various storage systems, including SQL and NoSQL databases. The tool supports distributed workers and features service discovery using Apache Zookeeper and Curator, along with advanced reporting capabilities. Users can define and manage benchmarking jobs using a simple JSON interface for job submission and monitoring.
Big Data DC - BenchPress
- 1. Palomino Labs, Inc.palominolabs.com Distributed Benchmarking with BenchPress
- 2. BenchPress • High performancebenchmarking • Distributed workers • Flexible • SQL databases • NoSQL datastores • DDOS • Anything (no, but most things)‽
- 3. Y!CSB, The Grinder,JMeter • Limited workload configuration • For storage systems only (Y!CSB) • Single host only
- 4. BenchPress Communication • ServiceDiscovery • Apache ZooKeeper • Apache Curator
- 5. BenchPress Communication • ServiceDiscovery • Apache ZooKeeper • Apache Curator • Job Control • Jetty • Jersey • Jackson
- 6.
- 7.
- 8.
- 9. • Submitting job •curl -X POST -H "Content-Type: application/json" • -d @job.json http://benchpress.foo:7000/controller/job • Job status & completion • curl http://benchpress.foo:7000/controller/job/<job-id>
- 10.
- 11.
- 12. See Also • ServiceDiscovery with Curator http://blog.palominolabs.com/2012/08/14/using-netflix-curator-for-service-discovery/ • Apache (née Netflix) Curator http://curator.incubator.apache.org/ • Yahoo! Cloud System Benchmark https://github.com/brianfrankcooper/YCSB/ • Code highlighting by http://markup.su/highlighter/
- 13. Palomino Labs, Inc.palominolabs.com Drew Stephens drew@palominolabs.com @dinomite http://www.slideshare.net/dinomite/big-data-dc-benchpress
Editor's Notes
- #2 Principal at Palomino Labs; software development consultancy Desktop webapps, native mobile apps, low level systems programming, and big data Also process consulting, focusing on making teams more efficient
- #4 Limited flexibility: only do one thing Yahoo! Cloud System Benchmark Only single host
- #5 Architecture … Curator, which Apache gobbled up from Netflix
- #7 - quanta: total number of operations - batchSize: per-thread, number of quanta to complete before an interstitial action; more later - Content of each action: configurable key & value generators
- #8 - Creating new objects for each thread - MongoDB instance - Key & value generators - quantaPerThread: total number of things to do batchSize…see Runnable
- #10 Easy to integrate with external tools! Because JSON
- #11 DIY statistics