Palomino Labs, Inc. palominolabs.com
Distributed Benchmarking with
BenchPress
BenchPress
• High performance benchmarking
• Distributed workers
• Flexible
• SQL databases
• NoSQL datastores
• DDOS
• An...
Y!CSB, The Grinder, JMeter
• Limited workload configuration
• For storage systems only (Y!CSB)
• Single host only
BenchPress Communication
• Service Discovery
• Apache ZooKeeper
• Apache Curator
BenchPress Communication
• Service Discovery
• Apache ZooKeeper
• Apache Curator
• Job Control
• Jetty
• Jersey
• Jackson
BenchPress Job Definition
MongoDBTaskFactory
MongoDBRunnable
• Submitting job
• curl -X POST -H "Content-Type: application/json"
• -d @job.json http://benchpress.foo:7000/controller/j...
Advanced Reporting
Advanced Reporting
See Also
• Service Discovery with Curator
http://blog.palominolabs.com/2012/08/14/using-netflix-curator-for-service-discov...
Palomino Labs, Inc. palominolabs.com
Drew Stephens
drew@palominolabs.com
@dinomite
http://www.slideshare.net/dinomite/big-...
Upcoming SlideShare
Loading in …5
×

Big Data DC - BenchPress

675 views

Published on

Talk at Big Data DC (http://www.meetup.com/bigdatadc/) about BenchPress (https://github.com/palominolabs/benchpress)

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
675
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Principal at Palomino Labs; software development consultancy Desktop webapps, native mobile apps, low level systems programming, and big data Also process consulting, focusing on making teams more efficient
  • Limited flexibility: only do one thing Yahoo! Cloud System Benchmark Only single host
  • Architecture … Curator, which Apache gobbled up from Netflix
  • - quanta: total number of operations - batchSize: per-thread, number of quanta to complete before an interstitial action; more later - Content of each action: configurable key & value generators
  • - Creating new objects for each thread - MongoDB instance - Key & value generators - quantaPerThread: total number of things to do batchSize…see Runnable
  • Easy to integrate with external tools! Because JSON
  • DIY statistics
  • Big Data DC - BenchPress

    1. 1. Palomino Labs, Inc. palominolabs.com Distributed Benchmarking with BenchPress
    2. 2. BenchPress • High performance benchmarking • Distributed workers • Flexible • SQL databases • NoSQL datastores • DDOS • Anything (no, but most things)‽
    3. 3. Y!CSB, The Grinder, JMeter • Limited workload configuration • For storage systems only (Y!CSB) • Single host only
    4. 4. BenchPress Communication • Service Discovery • Apache ZooKeeper • Apache Curator
    5. 5. BenchPress Communication • Service Discovery • Apache ZooKeeper • Apache Curator • Job Control • Jetty • Jersey • Jackson
    6. 6. BenchPress Job Definition
    7. 7. MongoDBTaskFactory
    8. 8. MongoDBRunnable
    9. 9. • Submitting job • curl -X POST -H "Content-Type: application/json" • -d @job.json http://benchpress.foo:7000/controller/job • Job status & completion • curl http://benchpress.foo:7000/controller/job/<job-id>
    10. 10. Advanced Reporting
    11. 11. Advanced Reporting
    12. 12. See Also • Service Discovery with Curator http://blog.palominolabs.com/2012/08/14/using-netflix-curator-for-service-discovery/ • Apache (née Netflix) Curator http://curator.incubator.apache.org/ • Yahoo! Cloud System Benchmark https://github.com/brianfrankcooper/YCSB/ • Code highlighting by http://markup.su/highlighter/
    13. 13. Palomino Labs, Inc. palominolabs.com Drew Stephens drew@palominolabs.com @dinomite http://www.slideshare.net/dinomite/big-data-dc-benchpress

    ×