Load testing with Blitz
Upcoming SlideShare
Loading in...5
×
 

Load testing with Blitz

on

  • 995 views

We all know that load testing is important, but it's all too common that it's left to the very end of a project and it's invariably the first thing that gets dropped when budgets and timeframes get ...

We all know that load testing is important, but it's all too common that it's left to the very end of a project and it's invariably the first thing that gets dropped when budgets and timeframes get cut. Furthermore, most of us don't know where or how to start implementing effective load tests, let alone how to analyse the results.

Lindsay Holmwood, Software Manager at Bulletproof Networks, will be talking about integrating performance testing into your application development + deploy cycle from the very beginning, using inexpensive and easy to use SaaS tools.

There will be a hands on demonstration of the Blitz load + performance testing tool, coupled with a brief dive into the Blitz API internals to retrieve and analyse advanced reporting information.

Statistics

Views

Total Views
995
Views on SlideShare
986
Embed Views
9

Actions

Likes
1
Downloads
9
Comments
0

3 Embeds 9

http://bottlenose.com 4
http://us-w1.rockmelt.com 3
http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Load testing with Blitz Load testing with Blitz Presentation Transcript

  • Load testing
  • Who is this dude?
  • Lindsay Holmwood
  • BULLETPROOF
  • I Run this thing
  • Have tried and failed manytimes to implement effective load testing
  • A story
  • New project
  • Replacing a high- profile website (new infrastructure, new application)
  • 1000concurrent users
  • “We need 100% uptime”
  • Project starts
  • Requirements change
  • Deadlinesaren’t met
  • Performance testing? Fuck it! We’ll do it live!
  • Performance testing?“You have one day”
  • Problem:
  • Treated asWAterfall-esque QA
  • It’s anon-functional requirement
  • WHYDo we load test?
  • Capacityplanning
  • “Can the site deal with load?”
  • Defectdetection
  • “Is this codegoing to cripple the site?”
  • Optimisation
  • “What happens if we change this Setting?”
  • Historical analysis
  • “Is the sitegetting slower?”
  • Cache warming
  • “Prepare theinfrastructure for Failover”
  • WHYDo we load test?
  • A combination of proactive &Reactive reasons
  • A combination of proactive &Reactive reasons
  • It’s anon-functional requirement
  • Why does load testing FAIL?(from a technical perspective)
  • Hownot theWHAT
  • TooL fetishism
  • Get bogged down in details
  • how the testinginfrastructure works
  • Micro vsMacro
  • Business Does not careabout your tools
  • Businesscares About:
  • “Can the site deal with load?”
  • Recommendation:
  • Use tool withlowest barrier TO entry
  • Blitz
  • Runs on AWS
  • Web form
  • Browser plugins Firefox, Chrome
  • REST API
  • API ClientsRuby, Python, Java Node.JS, Perl
  • CommonCommand Line
  • Excellentdocumentation docs.blitz.io
  • Inexpensive 250 concurrent usersover 60 second period == free
  • SPRINTS &RUSHES
  • Let’s dive in!
  • Sign up @ http://blitz.io/
  • $ gem install blitz$ blitz helpUsage: blitz <command> <options> help - Display this help account:about - Show information about your account api:init - Validate and login with your API key couch:fuzz - Auto generate blitz tests from CouchDB curl - Run a sprint or a rush curl:help - Show help on sprint and rushing traceroute - Run traceroute remotely traceroute:help - Show help on traceroute version - Show the version of this Ruby gem$ blitz api:init# writes credentials to ~/.blitz/credentials
  • $ blitz curl:helpUsage: blitz curl <options> <url>--user-agent -A <string> User-Agent to send to server--cookie -b name=<string> Cookie to send to the server (multiple)--data -d <string> Data to send in a PUT or POST request--dump-header -D <file> Print the request/response headers--referer -e <string> Referer URL--help -h Help on command line options--header -H <string> Custom header to pass to server--pattern -p <s>-<e>:<d> Ramp from s to e concurrent requests in d secs--region -r <string> california|oregon|virginia|singapore|ireland|japan--status -s <number> Assert on the HTTP response status code--timeout -T <ms> Wait time for both connect and responses--user -u <user[:pass]> User and password for authentication--request -X <string> Request method to use (GET, HEAD, PUT, etc.)--variable -v <string> Define a variable to use--verbose -V Print the request/response headers--tlsv1 -1 Use TLSv1 (SSL)--sslv2 -2 Use SSLv2 (SSL)--sslv3 -3 Use SSLv3 (SSL)
  • $ blitz curl --region singapore --pattern 1-250:20 --timeout 10000 --verbose http://example.org/
  • user growth / time$ blitz curl --region singapore --pattern 1-250:20 --timeout 10000 --verbose http://example.org/
  • user growth / time for AWS lag$ blitz curl --region singapore --pattern 1-250:20 --timeout 10000 --verbose http://example.org/
  • user growth / time for AWS lag$ blitz curl --region singapore --pattern 1-250:20 --timeout 10000 --verbose http://example.org/rushing from singapore... Time Users Response Hits Timeouts Errors Hits/s Mbps 2.5s 31 -1.000s 0 0 0 8.3s 103 1.075s 146 0 9 50.53 0.88 11.2s 140 0.406s 376 0 9 79.64 1.41 14.1s 176 0.409s 682 0 9 105.24 1.86 22.8s 0 0.417s 1552 0 9 15.57 0.27
  • --pattern 1-250:20
  • blitz calculates the growth automatically--pattern 1-250:20
  • Sawtooth--pattern 1-250:20,1-250:20,1-250:20
  • TV AD / Twittersplosion --pattern 200-5000:30,5000-400:60
  • Integrate with yourdevelopment cycle
  • VCS CIDeploy automation
  • VCS CIDeploy automation
  • gitpost-receive hook
  • GitHub web hooks
  • Doesn’t take into account test failures
  • Is the code actually deployed?
  • VCS CIDeploy automation
  • CRON JOB
  • JENKINS JOB
  • GREAT if you have a CI system
  • Runs tests butSeparate system
  • What if you’re makinginfrastructure changes during a load test?
  • VCS CIDeploy automation
  • Fabric
  • Capistrano
  • Run tests when you deploy
  • Use the same toolchain
  • Testing lives IN the app.
  • What about triggering load tests on infrastructure changes?
  • MVP
  • Just getsomething working!
  • Problem:
  • No REporting :-(
  • THIS IS ITrushing from singapore... Time Users Response Hits Timeouts Errors Hits/s Mbps 2.5s 31 -1.000s 0 0 0 8.3s 103 1.075s 146 0 9 50.53 0.88 11.2s 140 0.406s 376 0 9 79.64 1.41 14.1s 176 0.409s 682 0 9 105.24 1.86 22.8s 0 0.417s 1552 0 9 15.57 0.27
  • SOL ON:
  • Optimisation
  • “What happens if we change this Setting?”
  • Historical analysis
  • “Is the sitegetting slower?”
  • Showstopper?
  • ...
  • $ gem install blitz# ...rushing from singapore... Time Users Response Hits Timeouts Errors Hits/s Mbps 2.5s 31 -1.000s 0 0 0 8.3s 103 1.075s 146 0 9 50.53 0.88 11.2s 140 0.406s 376 0 9 79.64 1.41 14.1s 176 0.409s 682 0 9 105.24 1.86 22.8s 0 0.417s 1552 0 9 15.57 0.27
  • Can we duckpunch our way to success?
  • #!/usr/bin/env ruby# load_test.rbrequire blitzarguments = "--region singapore --pattern 1:250:60"url = "http://example.org/"command = "#{arguments} #{url}"job = Blitz::Curl.parse(command)curl = Blitz::Command::Curl.newcurl.rush(job)
  • # ruby load_test.rbrushing from singapore... Time Users Response Hits Timeouts Errors Hits/s Mbps 2.5s 31 -1.000s 0 0 0 8.3s 103 1.075s 146 0 9 50.53 0.88 11.2s 140 0.406s 376 0 9 79.64 1.41 14.1s 176 0.409s 682 0 9 105.24 1.86 22.8s 0 0.417s 1552 0 9 15.57 0.27
  • #!/usr/bin/env ruby# load_test.rbrequire blitzarguments = "--region singapore --pattern 1:250:60"url = "http://example.org/"command = "#{arguments} #{url}"job = Blitz::Curl.parse(command)curl = Blitz::Command::Curl.newcurl.rush(job)
  • #!/usr/bin/env ruby# load_test.rbrequire blitzarguments = "--region singapore --pattern 1:250:60"url = "http://example.org/"command = "#{arguments} #{url}"job = Blitz::Curl.parse(command)curl = Blitz::Command::Curl.newcurl.rush(job)
  • Blitz::Curl.parse(command) returns Blitz::Curl::Rush
  • #!/usr/bin/env ruby# load_test.rbrequire blitzarguments = "--region singapore --pattern 1:250:60"url = "http://example.org/"command = "#{arguments} #{url}"job = Blitz::Curl.parse(command)curl = Blitz::Command::Curl.newcurl.rush(job)
  • Blitz::Command::Curl#rush
  • initiates the test + outputs the test resultsBlitz::Command::Curl#rush
  • #!/usr/bin/env ruby# load_test.rbrequire blitzarguments = "--region singapore --pattern 1:250:60"url = "http://example.org/"command = "#{arguments} #{url}"job = Blitz::Curl.parse(command)curl = Blitz::Command::Curl.newcurl.rush(job)puts job.result.timeline.to_json
  • Produces muck
  • duck punch ourway to success
  • class Blitz::Curl::Rush class Point def to_json(*args) { timestamp => @timestamp, duration => @duration, total => @total, hits => @hits, errors => @errors, timeouts => @timeouts, volume => @volume, txbytes => @txbytes, rxbytes => @rxbytes, steps => @steps, }.to_json end end # ...
  • # ... class Step def to_json(*args) { duration => @duration, connect => @connect, errors => @errors, timeouts => @timeouts, asserts => @asserts, }.to_json end endend
  • #!/usr/bin/env ruby# load_test.rbrequire blitzarguments = "--region singapore --pattern 1:250:60"url = "http://example.org/"command = "#{arguments} #{url}"job = Blitz::Curl.parse(command)curl = Blitz::Command::Curl.newcurl.rush(job)puts job.result.timeline.to_json
  • {"start":1335694346,"finish":1335694367,"results":[{"timestamp":2.502626,"duration":1.13912,"total":1,"hits":1,"errors":0,"timeouts":0,"volume":2,"txbytes":258.0,"rxbytes":17810.0,"steps":[{"duration":2.13912,"connect":0.260004,"errors":0,"timeouts":0,"asserts":0}]},{"timestamp":5.016102,"duration":0.640047,"total":5,"hits":5,"errors":0,"timeouts":0,"volume":4,"txbytes":1548.0,"rxbytes":90882.0,"steps":[{"duration":1.640047,"connect":0.158944,"errors":0,"timeouts":0,"asserts":0}]},{"timestamp":7.524594,"duration":0.639987,"total":10,"hits":10,"errors":0,"timeouts":0,"volume":5,"txbytes":3354.0,"rxbytes":182222.0,"steps":[{"duration":1.639987,"connect":0.158999,"errors":0,"timeouts":0,"asserts":0}]},{"timestamp":10.03388,"duration":0.63961,"total":20,"hits":20,"errors":0,"timeouts":0,"volume":7,"txbytes":5418.0,"rxbytes":364902.0,"steps":[{"duration":1.63961,"connect":0.158847,"errors":0,"timeouts":0,"asserts":0}]},{"timestamp":12.543211,"duration":0.640421,"total":31,"hits":31,"errors":0,"timeouts":0,"volume":9,"txbytes":9030.0,"rxbytes":565850.0,"steps":[{"duration":1.640422,"connect":0.158767,"errors":0,"timeouts":0,"asserts":0}]},{"timestamp":15.051705,"duration":0.639894,"total":44,"hits":44,"errors":0,"timeouts":0,"volume":10,"txbytes":12384.0,"rxbytes":803334.0,"steps":[{"duration":1.639894,"connect":0.158794,"errors":0,"timeouts":0,"asserts":0}]},{"timestamp":17.560136,"duration":-1.0,"total":44,"hits":44,"errors":0,"timeouts":0,"volume":0,"txbytes":12384.0,"rxbytes":803334.0,"steps":[{"duration":0.0,"connect":0.0,"errors":0,"timeouts":0,"asserts":0}]}]}
  • Do with asyou please
  • Here’s one Iprepared earlier..
  • ...
  • You don’t have to set up a tool
  • But you have towrite reporting?
  • At least you have load testing?
  • It’s on the way..
  • http://support.blitz.io/discussions/questions/114-sharing-reports
  • Where tofrom here?
  • Pull in data from other places
  • new relicGanglia/collectd Git commits
  • Github:square/crossfilter
  • Don’t like Blitz?
  • SaaS Load impactBrowser mob
  • Open Source Tsung JMeter
  • Start Using something!
  • Low barrier of entry is key
  • Integrate with your existing dev tools
  • Questions?