1.
Performance Testing
Apache Benchmark & JMeter
Antoni Orfin
antoniorfin@gmail.com

2.
Performance Testing
Types of testing
Stress test1
Load test2
- Run test from low to high load
- Find the peak for the system
„If we reach more than 100 concurrent
users, the system is slowing down”
- Break the system
- Test if it fails and recovers gracefully (recoverability)

3.
Performance Testing
Apache Benchmark
Download Apache Utils1
Run test2
$ apt-get install apache2-utils
$ ab -n 100 -c 1 http://localhost/
$ ab -n 100 -c 50 http://localhost/
-n requests Number of requests to perform
-c concurrency Number of multiple requests to make at a time
Concurrency does not mean number of
simultaneous users!

4.
Performance Testing
Apache Benchmark
Interpret results3
Server Software: nginx/1.6.2
Server Hostname: localhost
Server Port: 80
Document Path: /
Document Length: 94873 bytes
Concurrency Level: 50
Time taken for tests: 0.094 seconds
Complete requests: 100
Failed requests: 7 (Connect: 0, Receive: 0, Length: 7, Exceptions: 0)
Total transferred: 9503493 bytes
HTML transferred: 9487293 bytes
Requests per second: 1064.54 [#/sec] (mean)
Time per request: 46.969 [ms] (mean)
Time per request: 0.939 [ms] (mean, across all concurrent requests)
Transfer rate: 98797.65 [Kbytes/sec] received

5.
Performance Testing
Apache Benchmark
Load testing4
$ ab -n 10000 -c 10 http://localhost/ # c to low (server is not
$ ab -n 10000 -c 100 http://localhost/ # saturated, response
$ ab -n 10000 -c 250 http://localhost/ # times are stable)
$ ab -n 10000 -c 500 http://localhost/
$ ab -n 10000 -c 750 http://localhost/ # c too high (server is
$ ab -n 10000 -c 1000 http://localhost/ # saturated, response)
times are increasing)
HERE J

6.
Performance Testing
Apache Benchmark
Cons-
- Tests only one URL at the same time.
- Running load test with various concurrency and
collecting results into nice graphs is irritating
- Can’t run distributed test (with multiple test servers)
Pros-
- Zero-configuration
- Easy to learn and to run first test
- Small CPU/memory footprint

7.
Performance Testing
Apache JMeter
Download JMeter1
Download JMeter Plugins2
http://jmeter.apache.org/download_jmeter.cgi
Plugin manager: https://jmeter-plugins.org/install/Install/
Plugins, mostly:
https://jmeter-plugins.org/wiki/Start/#Thread-Groups
https://jmeter-plugins.org/wiki/Start/#Graphs

8.
Performance Testing
Apache JMeter
Sample load-test scenario3

9.
Performance Testing
Apache JMeter
Definining parameters4
- Define variables in „User Defined Variables”
- Allow to pass variables via CLI
${__P(host,${host})} - will use value from „host” CLI
argument or from ”User Defined Variables” if not
passed.
$ ./bin/jmeter -t scenario.jmx -Jhost my-host.com
HINT

10.
Performance Testing
Apache JMeter
Using CSV file with URLs5
- You can test multiple URLs in a single load-test
- Group results by categorizing URLs
Sample CSV:
Homepage,/
Category,/wallmurals
Category,/prints-and-posters
ProductPage,/wallmurals/cat-425225252
ProductPage,/stickers/dog-12789

11.
Performance Testing
Apache JMeter
Defining threads scenario6
- Use „Concurrency Thread Group” (from JMeter
Plugins) to test how your website behaves under
increasing number of threads

12.
HINT
Performance Testing
Apache JMeter
Making HTTP Request7
- As a „Name” use variable from CSV file (that will group results)
- You can include also other parameters in CSV (e.g. method,
protocol)
- To avoid network latencies use HEAD HTTP method
- server will return empty responses
- may depend on your application/server’s configuration

13.
Performance Testing
Apache JMeter
Moving „Concurrent Threads” into Real Users8
- Find out Average Time on Page (not average session) in your
Google Analytics
- Use „Gaussian Random Timer” to add some randomized delay
after each request

14.
Performance Testing
Apache JMeter
Getting statistics from Google Analytics9
Nb of concurrent users:
concurrent_users = (peak_hourly_visits * average_session_duration) / 3600
e.g. 540 * 201 sec / 3600 sec = 30 users
Peak traffic: (peak_hourly_pageviews / 3600)
e.g. 21000 / 3600 = 5,83 req/s
Peak
21.000 pageviews

15.
Performance Testing
Apache JMeter
Running test10
$ ./bin/jmeter -t scenario.jmx -n –l results.jtl
Collecting results11
- Always run tests from CLI (non-interaction mode) to avoid
memory/CPU problems
- Use „Graphs Generator” (from JMeter plugins) that
automatically saves graphs after the test.
- …or preview results online: https://sense.blazemeter.com/gui/

16.
Performance Testing
Apache JMeter
Cons-
- Difficult to configure (Java & JMeter needs to be
properly tuned-up)
- Sometimes results are not so straightforward to
interpret
Pros-
- Can test multiple URLs in a single load-test
- Can run distributed tests, even in cloud
(e.g. www.blazemeter.com)
- Can be easily integrated into CI (Jenkins plugin)
- Allows to compose complex scenarios, even with Selenium

17.
Performance Testing
General hints
- Different machines: Never run tests on the same machine that
application is running
- The same datacenter: To avoid network latencies (ping), it’s better
to run tests from the same datacenter as the target application.
- Rent cloud with hourly pricing: Amazon EC2 „on-demand”
instances are great for short load test:
1. Prepare your test scenario and JMeter installation
2. Rent EC2 instance just for the time that will be needed to finish
the test.
- Watch out for production infrastructure:
1. If you don’t have separate infrastructure to test (cloned
production), run tests during the lowest traffic (e.g. at night).
2. 80% probability that you will take-down the application during
the load test.

18.
Contact me at:
antoniorfin@gmail.com
linkedin.com/in/antoniorfin
twitter.com/antoniorfin
www.pixersize.com
Thank you!
Questions & Answers