• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Big Data DC - BenchPress
 

Big Data DC - BenchPress

on

  • 621 views

Talk at Big Data DC (http://www.meetup.com/bigdatadc/) about BenchPress (https://github.com/palominolabs/benchpress)

Talk at Big Data DC (http://www.meetup.com/bigdatadc/) about BenchPress (https://github.com/palominolabs/benchpress)

Statistics

Views

Total Views
621
Views on SlideShare
613
Embed Views
8

Actions

Likes
0
Downloads
4
Comments
0

1 Embed 8

https://twitter.com 8

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Principal at Palomino Labs; software development consultancy Desktop webapps, native mobile apps, low level systems programming, and big data Also process consulting, focusing on making teams more efficient
  • Limited flexibility: only do one thing Yahoo! Cloud System Benchmark Only single host
  • Architecture … Curator, which Apache gobbled up from Netflix
  • - quanta: total number of operations - batchSize: per-thread, number of quanta to complete before an interstitial action; more later - Content of each action: configurable key & value generators
  • - Creating new objects for each thread - MongoDB instance - Key & value generators - quantaPerThread: total number of things to do batchSize…see Runnable
  • Easy to integrate with external tools! Because JSON
  • DIY statistics

Big Data DC - BenchPress Big Data DC - BenchPress Presentation Transcript

  • Palomino Labs, Inc. palominolabs.com Distributed Benchmarking with BenchPress
  • BenchPress • High performance benchmarking • Distributed workers • Flexible • SQL databases • NoSQL datastores • DDOS • Anything (no, but most things)‽
  • Y!CSB, The Grinder, JMeter • Limited workload configuration • For storage systems only (Y!CSB) • Single host only
  • BenchPress Communication • Service Discovery • Apache ZooKeeper • Apache Curator
  • BenchPress Communication • Service Discovery • Apache ZooKeeper • Apache Curator • Job Control • Jetty • Jersey • Jackson
  • BenchPress Job Definition
  • MongoDBTaskFactory
  • MongoDBRunnable
  • • Submitting job • curl -X POST -H "Content-Type: application/json" • -d @job.json http://benchpress.foo:7000/controller/job • Job status & completion • curl http://benchpress.foo:7000/controller/job/<job-id>
  • Advanced Reporting
  • Advanced Reporting
  • See Also • Service Discovery with Curator http://blog.palominolabs.com/2012/08/14/using-netflix-curator-for-service-discovery/ • Apache (née Netflix) Curator http://curator.incubator.apache.org/ • Yahoo! Cloud System Benchmark https://github.com/brianfrankcooper/YCSB/ • Code highlighting by http://markup.su/highlighter/
  • Palomino Labs, Inc. palominolabs.com Drew Stephens drew@palominolabs.com @dinomite http://www.slideshare.net/dinomite/big-data-dc-benchpress