Methodology of load testing of a containerised Java application. What is load testing? How to properly load test an application? Troubleshooting of common issues through metrics.

1. Load Testing
for containerized java

2. Load testing for containerized java
Prerequisites
Memory model knowledge
Docker knowledge
Web protocol knowledge

3. Welcome!

4. Load testing for containerized java
docker & docker-compose
glebiller/local-java-load-testing.git
Logistics
#tvld-cop-qa

5. Load testing for containerized java
4
3
2
1
Agenda
Load
Testing?
Case
study
Common
issues
Testing
method

6. Load Testing?

7. Modeling a system behavior
under load and constraints

8. Overloading
Standard use
High concurrency

9. Load testing for containerized java
Type of load tests
Load Tests
Smoke
tests
Performance
tests
Stress
tests
Soak
tests
Spike
tests

10. Testing
method

11. Configure. Measure. Analyse.

12. System constraints are
the most important!

14. Load testing for containerized java
Systems
Java Virtual Machine
Server / Container
Tomcat / Middleware

15. Load testing for containerized java
Tomcat / Middleware
Request processor
HTTP thread pool
Data source pool

16. Load testing for containerized java
Java Virtual Machine
Heap memory
Memory ratio
Garbage collection

17. Load testing for containerized java
Server / Container
CPU
Memory
Disk & I/O

18. Load testing for containerized java
Concurrent users
Hourly sessions x Average session duration / 3600
12700 500s
x / 3600
= ~1700 users
Traveldoo — Nov 2020

19. Measure. Analyse.

20. Measure everything!

21. Load testing for containerized java
System constraints
CPU Memory
Data Sources
Disk I/O
Threads
Network

22. Load testing for containerized java
Tests results
Throughput Error rate
Latency Availability

23. Load testing for containerized java
Which monitoring or testing
tool are you familiar with?

24. VisualVM
the swiss army knife tool

25. Load testing for containerized java
VisualVM
https://visualvm.github.io/
Plugins to enable:
VisualVM-Extensions
VisualVM-Security
VisualVM-MBeans
Threads Inspector

26. Load testing for containerized java
JVM resources

27. Load testing for containerized java
CPU Sampling

28. Load testing for containerized java
MBeans

29. Analyse.

30. Load testing for containerized java
Let’s talk about statistics
Min Mean Max
Median
Percentile

31. Load testing for containerized java
Why use percentiles?
50th
90th
mean
50th
or median: 50% of the requests are faster than 155ms

32. Load testing for containerized java
Why use percentiles?
mean
50th
15th
50th
or median: 50% of the requests are faster than 370ms
20th

33. Load testing for containerized java
Traveldoo — November 2020
POST
GET

34. SLI
SLO
SLA

35. All measurements
should be
Service Level Indicators

36. Load testing for containerized java
Service Level Objectives
The system should not produce more than 1% errors
90th
request response time should be below 200ms
99th
request response time should be below 800ms
A specific endpoint must respond within 300ms

37. <100 ms <1 s <10 s
imperceptible noticeable
feedback
required

38. Load testing for containerized java
Service Level Agreement
Throughput Error rate
Latency
90th <
200ms
95th <
500ms
< 1%
booking
> 10
vus
Client A

39. Load testing for containerized java
Service Level Agreement
Throughput Error rate
Latency
90th <
200ms
95th <
500ms
< 5%
all
< 1%
booking
> 10
vus
Client A Client B
> 50
vus

40. Case study

41. mock-load-testing
docker run -p 8080:8080 kissy/mock-load-testing:1.2.1

42. docker-compose up -d
The big picture
mock-load-testing
k6 Configure

45. docker-compose up -d
The big picture
mock-load-testing
telegraf
k6 Configure
Measure

46. docker-compose run k6 run /scripts/common-issues.js
The big picture
mock-load-testing
influxdb
telegraf
grafana
k6 Configure
Measure
Analyse

47. @ http://localhost:3000
Grafana
Dashboard

48. Load testing for containerized java
Load testing results

49. Load testing for containerized java
Load testing results

50. Common
issues

51. Baseline
Validate testing configuration & system
Explore maximum system capabilities

52. Load testing for containerized java
Baseline

53. Load testing for containerized java
Baseline

54. Datasource bottleneck
Throttled due to lack of datasource thread
and/or due to slow datasource response

55. Load testing for containerized java
Datasource thread exhaustion

56. Load testing for containerized java
Datasource thread exhaustion

57. Memory bottleneck
Throttled due to shortage of free heap memory

58. Load testing for containerized java
Heap memory exhaustion

59. Load testing for containerized java
Heap memory exhaustion

60. Compute bottleneck
Throttled due to shortage of
computing resources

61. Load testing for containerized java
CPU overload

62. Load testing for containerized java
GC bottleneck
Throttled due to long amount of time spent
cleaning heap memory

63. Load testing for containerized java
Garbage Collector overload

64. Summary
& beyond

65. Load testing for containerized java
Summary
4
3
2
1 Load
Testing
Case
study
Common
issues
Testing
method

66. Load testing for containerized java
What’s next?
Regularily monitor production for traffic changes
Integrate load testing into build/deployment workflow
Implement Service Level Ojectives as K6 Threshold
Complete with Front-End load testing