Big Data DC - BenchPress
•Download as PPT, PDF•
1 like•1,061 views
Talk at Big Data DC (http://www.meetup.com/bigdatadc/) about BenchPress (https://github.com/palominolabs/benchpress)
More Related Content
What's hot
What's hot (20)
Similar to Big Data DC - BenchPress
Similar to Big Data DC - BenchPress (20)
Recently uploaded
Recently uploaded (20)
Big Data DC - BenchPress
- 1. Palomino Labs, Inc. palominolabs.com Distributed Benchmarking with BenchPress
- 2. BenchPress • High performance benchmarking • Distributed workers • Flexible • SQL databases • NoSQL datastores • DDOS • Anything (no, but most things)‽
- 3. Y!CSB, The Grinder, JMeter • Limited workload configuration • For storage systems only (Y!CSB) • Single host only
- 4. BenchPress Communication • Service Discovery • Apache ZooKeeper • Apache Curator
- 5. BenchPress Communication • Service Discovery • Apache ZooKeeper • Apache Curator • Job Control • Jetty • Jersey • Jackson
- 9. • Submitting job • curl -X POST -H "Content-Type: application/json" • -d @job.json http://benchpress.foo:7000/controller/job • Job status & completion • curl http://benchpress.foo:7000/controller/job/<job-id>
- 12. See Also • Service Discovery with Curator http://blog.palominolabs.com/2012/08/14/using-netflix-curator-for-service-discovery/ • Apache (née Netflix) Curator http://curator.incubator.apache.org/ • Yahoo! Cloud System Benchmark https://github.com/brianfrankcooper/YCSB/ • Code highlighting by http://markup.su/highlighter/
- 13. Palomino Labs, Inc. palominolabs.com Drew Stephens drew@palominolabs.com @dinomite http://www.slideshare.net/dinomite/big-data-dc-benchpress
Editor's Notes
- Principal at Palomino Labs; software development consultancy Desktop webapps, native mobile apps, low level systems programming, and big data Also process consulting, focusing on making teams more efficient
- Limited flexibility: only do one thing Yahoo! Cloud System Benchmark Only single host
- Architecture … Curator, which Apache gobbled up from Netflix
- - quanta: total number of operations - batchSize: per-thread, number of quanta to complete before an interstitial action; more later - Content of each action: configurable key & value generators
- - Creating new objects for each thread - MongoDB instance - Key & value generators - quantaPerThread: total number of things to do batchSize…see Runnable
- Easy to integrate with external tools! Because JSON
- DIY statistics