This document discusses performance testing tools and techniques. It defines performance from the perspectives of developers, infrastructure, and end users. Key aspects covered include defining realistic user scenarios, available tools like JMeter, ApacheBench, Gatling and Locust, and the importance of continuous performance testing. The document recommends using the Apdex score as part of your definition of done, specifying good test scenarios, running tests simultaneously, choosing the right tool for your needs, and considering tools like Taurus that enable continuous performance testing.

1. Performance testing
with JMeter
2018

2. ● Introduction
● Definition of “performance”
● Definition of “user”
● Defining scenarios
● Available tools
● Demo: JMeter
● Continuous testing
● Q/A
TOC

3. Introduction
let’s get everybody into same page

4. Definition of “Performance”
what does it mean?

5. From developer perspective
How well software utilises:
● CPU
● Memory
● I/O

6. From infrastructure perspective
How:
● much it utilises network bandwidth
● much requests the infrastructure can serve
● many servers it needs
● warm/loud is it at server room

7. From end user perspective
How fast the service feels.

8. APDEX

9. Apdex Read more
Apdex.org
“Apdex is a numerical measure of user satisfaction
with the performance of enterprise applications.”
Based on three zones of application
responsiveness:
1. Satisfied
2. Tolerating
3. Frustrated

10. Satisfied Read more
Apdex.org
The user is fully productive.
This represents the time value which
users are not impeded by application
response time (“T”).

11. Tolerating Read more
Apdex.org
The user notices performance lagging
within responses greater than “T”
seconds, but continues the process.

12. Frustrated Read more
Apdex.org
Performance with a response time
greater than “F” seconds is
unacceptable, and users may
abandon the process.
Picture from Apdex.org

13. The Formula
ApdexT
=
Satisfied count +
Tolerating count
Total samples
2

14. Example
= 0,7560 +
30
2
100

15. Example
= 0,8060 +
40
2
100

16. Example
= 1,00100 +
0
2
100

17. What about concurrent users?
That is one of a number part of Apdex
score calculation as “Total samples”.
Now, let’s talk about concurrent users.

18. Definition of “User”
what does it mean?

19. Realistic user
An realistic user comes to the service:
● with specific task in mind
● might get attracted to another task
● with different devices
● having limited accessibility
● causing probably external API calls too
Best way to specify this is to check from analytics service.

20. Unrealistic user
This is not realistic:
● 1000 users load one page, or random pages
● Some tool used to run GET requests and not even loading
embedded resources such as JS/CSS and images
● Goes pages through without reading times

21. Defining scenarios
they’re coming! what they’ll do?

22. Example case: Food Standards Agency
“Our job is to use our expertise and influence so that people can trust
that the food they buy and eat is safe and honest.”
● Visitors may read news and alerts around food safety
● Guidance for business owners
● Information for visitors about food safety and hygiene
● Food hygiene ratings
● Reporting functionalities

23. User tasks in the service
● Checking hygiene rating for an restaurant
● Reading and subscribing news and alerts
● Seeking information about food hygiene and safety
● Reporting a food problem
● Giving feedback
● Finding contact details
● Finding business services
● Seeking scientific studies about certain topics

24. Visitor looking for a food rating
1. Visitor lands on a homepage
2. Clicks “Food hygiene ratings” link
3. Two segments:
○ 80% User types writes a search keyword and hits enter
○ 20% Opens advanced options and uses it along with a keyword
4. Visitor clicks to a search result

25. Visitor lands on a homepage

26. Clicks “Food hygiene ratings”

27. Searching (with keyword only, 80%)

28. Searching (with additional options, 20%)

29. Visitor clicks a search result

30. Visitor clicks a search result

31. ● Introduction
● Definition of “performance”
● Definition of “user”
● Defining scenarios
● Available tools
● Demo: JMeter
● Continuous testing
● Q/A
TOC

32. Other tools
No worker without hammer

33. http://httpd.apache.org/docs/2
.4/programs/ab.html
ApacheBench (ab) Good for..
testing HTTP server
static files deliveries
and for bandwidth
testing
● Command line tool
● Originally designed for Apache HTTP server,
but widely used for other servers too
● Single-threaded (might become the
bottleneck on high concurrency tests)
● Does not have any response body parsing
features
http://httpd.apache.org
/docs/2.4/programs/ab.
html

34. Gatling Good for..
Java based web
projects.
● Written in Scala
● Licensed in Apache License 2.0
● Has recording feature/tool
● Based in France, part of France Open Source
Taskforce
● Tests written in Java
https://gatling.io/

35. Good for..
Java based web
projects.
https://gatling.io/

36. Good for..
projects that needs
web user interface for
controlling tests.
https://locust.io/

37. Locust Good for..
projects that needs
web user interface for
controlling tests.
● Web UI
● Tests written in Python
● Supports distributed running machines, can
scale!
● “Slaves” has multiple implementations
(boomer, golang, your custom)
● License MIT
https://locust.io/

38. Locust Good for..
projects that needs
web user interface for
controlling tests.
https://locust.io/

39. Molotov Good for..
performing simple
Python based test
scripts
● Tests written in Python
● Simple Python based test framework
● Uses coroutines
● Funded by Mozilla
molotov.readthedocs.io

40. Good for..
performing simple
Python based test
scripts
molotov.readthedocs.io

41. LoadBooster Good for..
load testing web
applications.
● Simulates user actions at browser, not traffic
● SaaS service
● Run from NodeJS client cli
● Service scales up to 10k users
● Users CasperJS, possibility to share
behavioural and performance tests?
loadbooster.com

42. Good for..
load testing web
applications.
jmeter.apache.org

43. Apache JMeter Good for..
simulating traffic for
complex scenarios.
● Tests are created using GUI (Java), stored as
XML
● Has recording feature
● License Apache License 2.0
● Very backed piece of software
● Funded by lots of sponsors (Apache
Foundation’s sponsors)
jmeter.apache.org

44. Demo
OK, stop speaking and show me!

45. The results

46. The results

47. The results

48. The results

49. Findings examples
● All web servers were not in use, but configured
● Lock issues
● API’s getting too slow (timeouts)

50. Continuous testing
let’s make it definition of done!

51. Taurus
What?
Taurus is a side project
by BlazeMeter that is
aiming for continuous
testing.
● Great tool for running automation-friendly
continuous testing
● Can be provisioned locally or cloud
● Supports almost all tools mentioned in this
presentation (20 executors)
● Recommended for projects where
performance must be quaranteed (make it
part of your “Definition of Done”)

52. What?
Taurus is a side project
by BlazeMeter that is
aiming for continuous
testing.

53. What?
Taurus is a side project
by BlazeMeter that is
aiming for continuous
testing.
Picture source: gettaurus.org

54. Summary Did you like?
Please tweet and follow
us on Twitter!
@iMiksu
@Wunder_io
● Use Apdex score as part of your DoD
● Specify good scenarios for performance testing
● Run different scenarios simultaneously
● Pick the right tool, JMeter is not ideal for all
cases!
● Consider continuous testing too

55. Questions?
Did you like?
Please tweet and follow
us on Twitter!
@iMiksu
@Wunder_io

56. Thank you