Big Data DC - BenchPress
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Big Data DC - BenchPress

  • 735 views
Uploaded on

Talk at Big Data DC (http://www.meetup.com/bigdatadc/) about BenchPress (https://github.com/palominolabs/benchpress)

Talk at Big Data DC (http://www.meetup.com/bigdatadc/) about BenchPress (https://github.com/palominolabs/benchpress)

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
735
On Slideshare
727
From Embeds
8
Number of Embeds
1

Actions

Shares
Downloads
4
Comments
0
Likes
0

Embeds 8

https://twitter.com 8

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Principal at Palomino Labs; software development consultancy Desktop webapps, native mobile apps, low level systems programming, and big data Also process consulting, focusing on making teams more efficient
  • Limited flexibility: only do one thing Yahoo! Cloud System Benchmark Only single host
  • Architecture … Curator, which Apache gobbled up from Netflix
  • - quanta: total number of operations - batchSize: per-thread, number of quanta to complete before an interstitial action; more later - Content of each action: configurable key & value generators
  • - Creating new objects for each thread - MongoDB instance - Key & value generators - quantaPerThread: total number of things to do batchSize…see Runnable
  • Easy to integrate with external tools! Because JSON
  • DIY statistics

Transcript

  • 1. Palomino Labs, Inc. palominolabs.com Distributed Benchmarking with BenchPress
  • 2. BenchPress • High performance benchmarking • Distributed workers • Flexible • SQL databases • NoSQL datastores • DDOS • Anything (no, but most things)‽
  • 3. Y!CSB, The Grinder, JMeter • Limited workload configuration • For storage systems only (Y!CSB) • Single host only
  • 4. BenchPress Communication • Service Discovery • Apache ZooKeeper • Apache Curator
  • 5. BenchPress Communication • Service Discovery • Apache ZooKeeper • Apache Curator • Job Control • Jetty • Jersey • Jackson
  • 6. BenchPress Job Definition
  • 7. MongoDBTaskFactory
  • 8. MongoDBRunnable
  • 9. • Submitting job • curl -X POST -H "Content-Type: application/json" • -d @job.json http://benchpress.foo:7000/controller/job • Job status & completion • curl http://benchpress.foo:7000/controller/job/<job-id>
  • 10. Advanced Reporting
  • 11. Advanced Reporting
  • 12. See Also • Service Discovery with Curator http://blog.palominolabs.com/2012/08/14/using-netflix-curator-for-service-discovery/ • Apache (née Netflix) Curator http://curator.incubator.apache.org/ • Yahoo! Cloud System Benchmark https://github.com/brianfrankcooper/YCSB/ • Code highlighting by http://markup.su/highlighter/
  • 13. Palomino Labs, Inc. palominolabs.com Drew Stephens drew@palominolabs.com @dinomite http://www.slideshare.net/dinomite/big-data-dc-benchpress