Apache JMeter is a load testing tool for web client

1. ● Performance testing
• Load Testing
• StressTesting
● What is JMeter
● JMeter basic elements
● Building a Script
●Running a Script
●Recording a Script (Bootstrapping)
Sam Varadarajan, for IEJUG
Svaradar.tech@gmail.com
Agenda

2. Performance Testing

3. What is Performance testing?
Performance testing is performed to determine how a system performs in terms of
responsiveness and stability under a particular workload. It is typically non-functional,
meaning, we only care about load on the system, scalability etc. Actual functionality is
not tested in this phase. (That will be done in Functional Test phase).
The real goal is to find that maximum load that the system can handle without
performance degradation or crashes.
There are several types of performance testing, but 2 are important:
• Load/Capacity Testing
• Stress Testing
We will be focusing mainly on Load Testing in this presentation.

4. Load / Capacity Testing
•Load testing is the simplest form of performance
testing. A load test is usually conducted to
understand the behavior of the system under a
specific (expected) sustained load.
•The load can be the expected concurrent number
of users on the application performing a specific
number of transactions within the set duration.
•In simple terms, it is the maximum load the horse
can carry without it tripping or the carriage
toppling!
•During the tests we will collect the the response
times of all the important business critical
transactions. If the database, application server,
etc. are also monitored, then this simple test can
itself point towards any bottlenecks in the
application software.

5. Stress Testing
•Every system has a capacity limit. When the load
goes beyond the limit, the System starts responding
very slowly and even produce errors. we have seen
this in Databases, Network etc.
•It is more visible on a Web site, as number of users
can drastically (often unpredictably) increase,
depending on content. We saw this recently with
HealthCare.gov! Sometimes, websites crash due to
attacks such as denial of service attack.
•In simple terms, Stress on the system is that last
straw on the camel's back that broke it.
•The purpose of the stress testing is to find the
capacity limit of the system. We do this by increasing
the load on the system beyond a safe limit, so the
system will eventually crash. With the findings, we
can adjust the parameters to stretch this limit.

6. General Load Testing Script
Load Testing itself can broken down into following steps:
•Create Test (Can be recorded using some tool)
•Configure workload settings
•Add measurement hooks
•Data drive script (using variables and parameters to generalize it)
•Add logging
•Generate reports/charts from performance data collected.

7. Load Testing - Manually
This is how they did load testing before.
Just ak a bunch of users to come to a
training room. The users will start using
the application according to a script and
run simultaneously!
We probably generated a very weak load
and didn't really stress it either!

8. Load Testing Automated
•All or most of these steps can be automated using several different
programs/scripts.
•There are testing tools that have integrated these into tools called Load Testing
tools.
•Mercury Load Runner and Rational Rose tools are commercial products that have
been around for a long time.
•With the advent of Web applications, several new tools have appeared in both
commercial and open source arena. Following slide shows a list of Load testing
tools available.

9. Load Testing Tools
Tool Name Company Name Notes
Apache JMeter An Apache Jakarta open source
project
Java desktop application for load testing and performance measurement.
BlazeMeter BlazeMeter Ltd. BlazeMeter is a JMeter compatible, self-service, load testing platform.
Blitz Spirent Communications Blitz is a service for load and performance testing of websites, mobile, web
apps and REST APIs in the cloud.
Gatling Open Source JVM application with scenarios as code and portable HTML reports.
Loader.io SendGrid Labs Cloud based load testing service
Load Impact Load Impact - AB Cloud based performance testing SaaS tool.
LoadRunner HP Performance testing tool primarily used Licensed.
Load Test (included with
Soatest)
Parasoft Performance testing tool
loadUI SmartBear Software Cross-platform load testing tool
Login VSI Login VSI, Inc. Performance testing software for Windows based virtualized desktops.
Licensed.
NeoLoad Neotys Load testing tool for web and mobile applications.
OpenSTA Open System Testing Architecture Open source web load/stress testing application, licensed under the Gnu
GPL.
Rational Performance Tester IBM Eclipse based large scale performance testing tool
Silk Performer Borland Application performance tool with cloud and local virtual agents. Licensed
Test Studio Telerik Test Studio is a load testing tool
Visual StudioUltimate edition Microsoft Visual Studio Ultimate edition includes a load test tool
WebLOAD RadView Load testing tool for web and mobile applications Licensed.[3]
(Courtesy: Wikipedia)

10. JMeter

11. What is JMeter?
•JMeter is a free, open-source performance measurement tool
written in Java
•JMeter is built by the reputed Apache project community
• JMeter is available as a desktop Java application and can be run
in GUI and batch modes. It can also be run in distributed modes.
•It has a completely extensible architecture. It allows us to
develops various elements that can be reused in different parts
of a test plan. It also allows plugins to be developed, so the tool
can be further extended.

12. Why JMeter?
 It is an Open source tool.
 Can load and performance test many different server types:
 Web - HTTP, HTTPS
 SOAP
 Database via JDBC
 LDAP
 JMS
 Mail - POP3(S) and IMAP(S)
 User friendly GUI Design. Can be run in batch and distributed modes to handle more load.
 Full multithreading framework allows concurrent sampling by many threads and simultaneous
sampling of different functions by separate thread groups.
 Can mimick Web browser functionality including Caching and Cookies.
 Controllers are configurable and can monitor the server performance.
 Test results can be captured in various format like summary report, Graph, Aggregate report,
Aggregate graph, Results in tree and Results in Table. The results can be stored in files for later
review as well.
 Extensible architecture. Supports plugins, bean shell etc to add more functionality.

13. Installing Jmeter
• Install Java (1.5 or higher)
• Download JMeter http://jmeter.apache.org/download_jmeter.cgi
• Add path of java installation in environment path variables.

14. •A test plan describes a series of steps
JMeter will execute when run.
•A complete test plan will consist of one
or more Thread Groups, logic conrollers,
sample generating controllers, listeners,
timers, assertions, and configuration
elements.
JMeter Test Plan

15. Elements of a Test Plan
The Test Plan is the wrapper for the entire test suite. It wraps around
the following elements in an hierarchical fashion.
•Thread Group
This is really the logic unit of a load test. There can be many thread
groups in a Test Plan.
–Setup number of thread
–Set up ramp up period
–No. of times test executes
•Controllers
–Sampler (Send Request to Server)
–Logical Controller (Customize logic to send request); includes
Loop Controller and One Time Controller.
•Listener
Help to capture and display results in various formats. For e.g.,
–Graph Result
–View Results Tree or Table format and many more.
•Timers
–Add a delay to the next request
Continued….

16. Elements of Test Plans
• Assertions
– Allow you to assert fact about responses received from HTTP request
• Configuration Elements
– Allow you configure settings
• Preprocessor
– Execute prior to sampler request
• Post Processor
– Execute some action after sampler request

17. 1. Config Element
2. Pre-Processor
3. Timer
4. Sampler
5. Post-Processor
6. Assertion
7. Listener
Order of Execution

18. Building a Test Plan

19. •Thread group elements are the
beginning points of any test plan.
•All controllers and samplers must
be under a thread group.
•Each thread represents a user
setting a thread group to 500
simulates 500 users.
Thread Group

20. Thread Group Contd...

21. •Samplers tell JMeter to send requests to a server and wait for a response.
They are processed in the order they appear in the tree.
Sampler

22. Sampler Contd...

23. •Listeners can be used to display (or store) performance information JMeter collects during tests.
Listener

24. Listener Contd...

25. Listener Contd...

26. •Logic Controllers let you
customize the logic that
JMeter uses to decide
when to send requests.
•Logic Controllers can
change the order of
requests coming from their
child elements. They can
modify the requests
themselves, cause JMeter
to repeat requests, etc.
Logic Controller

27. •A configuration element
works closely with a
Sampler. Although it does
not send requests (except
for HTTP Proxy Server ), it
can add to or
modifyrequests.
Configuration Elements

28. Configuration Element
Contd...

29. Configuration Element
Contd...

30. •Assertions allow you to assert facts about
responses received from the server being tested.
• Using an assertion, you can essentially "test"
that your application is returning the results you
expect it to.
Assertions

31. Other Test Elements
● A Pre-Processor executes some action prior to a Sampler Request being made.
● A Post-Processor executes some action after aSampler Request has been made.
●A Timer will cause JMeter to delay a certain amount of time before each sampler
which is in its scope.

32. Variables
JMeter allows
parameters,
properties and
variables to keep the
test scripts
generalized.
Syntax for Variable:
${VARIABLE}
• Variables can be
declared and used in
successive layers.

33. JMeter Test Plan Completed

34. JMeter Test Plan Completed
• Once you completed, you can save the
test plan with a meaningful name.
• The script file generated is an XML file
and will be saved with a .JMX
extension.
• You can run this script using command
line mode as well, so you don't have to
load GUI.

35. Running a Test Plan

36. Running in Batch Mode
On command line, in the JMeterbin directory, you can type:
jmeter -n -t C:samload_testscriptsLT600_laweb2_test_syt6.jmx -j
C:samload_testlogjmeter_1212.overnight.log -l C:samload_testlogjmeter_1212.overnight.jtl
Where,
LT600_laweb2_test_syt6.jmx is the script I saved in the completion step above.
-n makes it non-GUI
-t name of the test script you want to use
-j for the JMeter.log (I change this to match my log file naming above)
-l for the JTL file - in XML format, has the same details we get in listeners above.
We may have some use for this, but for now I am just saving it.
You can put the above command inside a batch file to save typing!

37. Appendix

38. Non-Testing Elements

39. Non-Testing Elements Contd...
HTTP(S) Script Recording Element

40. Recording a Script
• Setup a non-test element, HTTP(S)
Test Script Recorder on the
workbench.
• Setup all the inclusion, exclusion lists
and other properties.
• Click on Start.
Now, JMeter is running as a local proxy
server. If you adjust PC's proxy setting
to use this as the proxy server, all
requests will pass through JMeter.

41. Recording a Script Contd...
Next you need to setup the browser to use our JMeter listener as a
local proxy

42. Recording a Script Contd...

43. Recording a Script Contd...
• Now that we have captured all the
pages to be visited, we can clean it all
up. This is a good place to start.
• Parametrize and add variables
wherever possible and move the pages
(samplers) to a Thread Group and build
up our test plan.

44. Recording a script Contd...• An easier way is to use BadBoy software to capture the pages and export to
JMeter. Download and install Badboy software from
http://www.badboy.com.au/download/send?svId=124

45. Recording a Script Contd...

46. Real life Test Scenario
• Our App Servers kept crashing on
heavy load days. We had to constantly
restart the servers, bringing down all
the web users along with it. After
making some changes to the
application, had to simulate the huge
load on the servers to test if it will
sustain the next tax season. After
reviewing various tools, we chose
JMEter.

47. Real life Test scenario
Contd...
• When we upgraded our infrastructure*
recently, we used JMEter to compare
old environment and new environment
for performance. We ran the same set
of scripts against old and new
environments to identify any
bottlenecks.
* We upgrade to PowerBuilder 12.5, Oracle to 11g, Windows servers to
Windows 2008.

48. Real life Test scenario
Contd...
• Our Web application did an overkill of logging in the
database. We had a maximum concurrent user limit
set to 300 in Oracle. We wanted to increase that.
• To help with the tweaking, I ran JMeter scripts that
pushed the EAServers to connect to the database
continuously to make the database run out of
handles - sort of stressing the database to its limits.
We were able to tweak the setting and retest for the
load condition.
• Eventually, I refactored the code to reduce these
loggings. Of course, JMeter came to the rescue
again!

49. Demo

50. Q&A