The document provides guidance on developing reliable load test scripts and scenarios. It discusses test data requirements, parameterizing dynamic values, proper use of HTTP protocol versus GUI-level scripting, handling asynchronous requests, implementing think time and pacing, validating scripts, and best practices for load scenario configuration. The goal is to explore main points around scripting best practices, validating load scripts thoroughly, and configuration best practices to build effective performance tests.

1. Productive way of load test development
for any system using any tool
Arjan van den Berg
www.performancetestexpert.nl

2. Generic approach to develop reliable load
script(s) and scenario(s)
www.performancetestexpert.nl

3. Context presentation (1/2)
 Presentation is focused on developing reliable HTTP protocol load scripts and scenarios
 Advice when to script on Browser / GUI level
 It’s related to performance test development & configuration so that my humble knowledge
and mistakes, from the past becomes knowledge for the future
…..a reliable load test script / scenario is crucial within every stage of the software
development cycle……and is often underestimated….
www.performancetestexpert.nl

4. Context presentation (1/2)

5. To explore the following main points:
1. “Scripting best practices”, for developing load scripts
 Test data needed for scripting,
 Web (2.0) UI user interface
 Introduction
 Using HTTP protocol
 Script on GUI-level
 Web services scripting
2. How to validate a load scripts/scenario thoroughly.
3. Load script/scenario configuration best practices.

6. Scripting best practices, for developing load scripts
Test data
www.performancetestexpert.nl

7. Test data (1/3)
 Identify Test scenarios and get functional understanding (happy) test flow(s)
 Identify test data for all Test scenarios
 Unique user ID’s / accounts, sufficient to handle all load profiles !!
 Create/Get production like test data
 Of course we can also use mock/stubs, so responses are based on a given request
 Creating test data for your load test is IMPORTANT
 Test data should be “linked” to user ID, so that we know that each user “sees” is own
data….

8. Test data (2/3)
 Examples:
#1 User has unique shopping articles , “correlated” to userID
#2 session replication , is session (data) handled correct under(high) load
#1 #2

9. Test data (3/3)
Testdata should be releated to userIds / Account.
So for instance don’t create a script like:
 User A: Order a smart watch
 User B: Order a smart watch
 User C: Order a smart watch
But , like:
 User A: Order a smart watch , shoes, <prod x>,
 User B: Order a laptop, smartphone, <prod y>,
 User C: Order a wireless mouse , <prod z>

10. Test data (3/3)
Releated to session data, an other example .
A user search action:
Usecase:
User A: Search for <birds>
User B: Search for <cars>
User C: Search for <product X>
Result:
User A: Search results for birds
User B: Search results for cars
User C: Search for <product X>
An example of what can happen in real life…
Usecase:
User A: Search for <birds>
User B: Search for <cars>
User C: Serach for <product X>
Result:
User A: Search results for cars
User B: Search results for birds
User C: Serach for <product X>
“Session
Data
issue””

11. Scripting best practices, for developing load scripts
Web (2.0) UI Interface
www.performancetestexpert.nl

12. Web (2.0 ) UI Interface - Introduction
 A lot of GUI frameworks today Angular , React, …
 Asynchronous website changes are common
 Sometimes also a persistent , real time bi-directional web-socket channel is used…
 Polling requests
 A lot off java script client-side
You may think how to record this on http protocol level, but bottom-line still a get/post
request/response is used to communicate with the server so for technical reasons you have
rarely to use GUI-level scripting ….

13. Scripting best practices, for developing load scripts
Using HTTP protocol
www.performancetestexpert.nl

14. Web (2.0 ) UI Interface – HTTP Protocol
 Start with simple user flows first
 Make sure the script doesn’t record unneeded URLs / get / post requests
 Identify ALL dynamic data (as the response from server) and correlate it !!
 Recording scripts 2 times and compare in detail all request or use commercial tools
features
 Get first the login session(Ids) / correlate the login sequence
 Parameterize also all dynamic data; dates , GUIDs, (epoch) timestamps, …
 Dynamic data is fetched with “expressions” from server response;
 Some load test tools supporting xpath , jsonpath and so on…my personal flavor is
regular expressions (with grouping) to get the correct response into a parameter,
because this works always

15. Web (2.0 ) UI Interface – HTTP Protocol
 Parameterize scripts to support dynamic data set.
 Dynamic data, every simulated user get different data; avoids responses from cache …
 Use also with a few transactions wrong test data values e.g.
 wrong phone number format;
 forget to fill an edit field; and so on…,
Test will be more realistic…else your test will be to clean…..
 Use proper checks / assertions for all steps involved in the transaction.
 Check on errors
 Check also on expected results
 Use also parameters for assertions, get also from test data table
 Absence of assertions might result in misleading results….

16. Web (2.0 ) UI Interface – HTTP Protocol
 Make sure that script can be executed against multiple environments
 Users rarely log out , so don’t assume always “clean logout” and design scripts like that ..
 Headers and cookies are handled automatically(almost) within any tool
 Configure the think - and pacing time in the script.
 Don’t use a constant think time value.
 Distributing think time values in a range .(e.g 5 sec +/- 50%)
 Implement think time and pacing time within your script with care, pay attention to
timing!!
 Pacing is a must ! Sometimes not implemented at all….

17. Pacing in more detail
 Think time and Pacing, both are a kind of “wait” or “delay” in terms of time but still both
differ from each other
 Pacing can be the delay between two iterations of test
Responsetimealways 10 sec ?
 What happens with TPS when response time is 5 sec ?
Yellow bar == pacing Time
 So use pacing to get “steady” load
 To compare load test results SUT, release x, y, z….. always use pacing!!
Thinktime + Round trip time <= Pacing !

18.  Keep load scripts simple
 Avoid custom flow logic within the script, it’s better to split it up into 2 separate
scripts
 Don’t try to write custom error handling within the scripts, e.g. retry save
 All tests scripts should be independent,
 If data between scripts needs to be shared , use a temp file /queue which all
scripts can access
 Avoid checks which not occur in real user scenario as much as possible , e.g
query with a script to check the database for “expected content”

19. Web (2.0 ) UI Interface – HTTP Protocol
 Take Special attention to repeated / similar looking requests
• If repeated / intermittent requests are observed “like polling/heartbeat pattern”
• delete all except one and put this in a loop;
• use a variable extractor to check the state/response
 Solution above can block the test flow, so if needed fork a separate thread
 If you observe /expect asynchronously requests in your recorded script
 Are maybe recorded sequential; but within a normal browser executed at the same
time
 Check async requests, with browser developer tools, e.g In Chrome Network TAB, XHR
 At least group such XMLHttpRequest together in your load script , or send them in
parallel

20. Web (2.0 ) UI Interface – HTTP Protocol
 Anyhow, even if the requests are send synchronously within your load script , it will still
work (almost everytime), but is related to timings less accurate….
 Pay attention to the transactions within your loadscript, should cover the right http
requests…
 If a websocket channel is used…
 Try to understand the (async) message patterns within this websocket channel
 Measuring response times accurate, can be challenging
 All requests/responses are send to one, channel, to take special attention to:
 Assertions & Extracting parameters needs to be (extreme) generic
 To get scripts up & running very often Multiple threads are needed,
 Debug / Maintenance websocket based tests can be (very) difficult…

21. Complex websocket script (Neoload)

22. XHR Requests example (send in parallel)

23. Scripting best practices, for developing load scripts
Script on GUI level
www.performancetestexpert.nl

24. Web (2.0 ) UI Interface – Script on GUI level
Scripting on GUI level remarks / some personal experiences
 GUI level scripting is for sure not record and playback !
 GUI level tests are used for backend loadtesting, frontend end-to-end user experience
testing is a different test !!
 GUI Object identification needs also a learning curve/ gain experience with it..
 GUI scripts also need maintenance…
 Popup windows can be an issue
 Assertions are also GUI objects, so needs attention !
 Errors needs also to be identified as a GUI object

25. Web (2.0 ) UI Interface – Script on GUI level
 Scripting on GUI level vs. HTTP
1. If possible use HTTP recording, and focus on fast script generation
 Framework parameters in Neoload
 Automatic correlation in other commercial tools
2. If the GUI is using Ajax “on average” still possible to use HTTP protocol
3. If the application is full websocket based, and the majority of the calls async. switch to
browser / GUI level
4. But to get GUI scripts stable takes also (development) time..
5. GUI Script flow is less technical, and more easy to explain to other stake holders
6. GUI script can reduce maintenance effort,

26. Scripting best practices, for developing load scripts
Webservices
www.performancetestexpert.nl

27. Webservices
 Webservices script
 Pretty straight forward
 Often a subset of existing functional tests can be used / converted to a performance
test (work in this area close together with the functional testautomation engineer)
 Use WSDL import within the load test tool, if possible, soapActions then available
 Test also error paths; wrong data format in request (small percentage e.g. 1%)
 Of course use assertions to check expected response
 Use also response size check (in bytes) to check expected response
 Importance of test data
 Size message , large included attachments (Important for parsing, memory usage,
Database request(s),…)
 Make sure that all operations / transformations are covered
 Test security impact in a separate test

28. How to validate a load scripts/scenario thoroughly.
www.performancetestexpert.nl

29. Validate loadscript and scenario
 Verify the scripts during design time
 Validate with one iteration with one user
 Validate with multiple iterations (#10) with one user
 Validate with multiple iterations (#10) with multiple concurrent users (#10)
DONT START TOO QUICKLY WITH A FULL LOADTEST

30. Validate loadscript and scenario
 Important best practices during validation script
 Execute verification with full log in load test tool
 Checking parameter/test data values during execution
 Check response against recorded session
 Check server logs during replay, or if possible check sessions in APM tools ; check
together with developers that what you see in the log is correct / valid
 Check if pacing / think time works as expected
 Check application server (e.g jvm) and resource usage SUT
 If test with one user is already (very) slow, report a performance problem….

31. Load script/scenario configuration best practices
www.performancetestexpert.nl

32. loadscenario
 Load scenario configuration (1/2)
 Group user scenarios / configurate the ratio between scripts
 User standard naming conventions for the different load profiles e.g. Baseline, Stress,
Load, Spike test
 Keep the settings for the load profiles fixed during the different test cycles, else you
loose the possibility to compare results…, because you have a lot of options there
 Rampup time, ramp down, duration;
 browser cache settings;
 static content on/of;
 rendezvous points;
 Pacing time, Think time(s);
 Option X,Y,Z

33. loadscenario
 Load scenario configuration (2/2)
 In case of (script) error, stop de Virtual user and start a new one,
 To avoid collateral damage
 If one step fails, there is a high risk that all other steps fail
 Does not appy for webservices…
 If possible make notes within the testrun results settings
 what the run conditions are;
 Even better automate this with a CI/CD pipeline
 Store at least SUT software version(s);
 And e.g Threadpool settings DB pool, log level SUT ,

34. A Quick Recap
 Test data must be in place before developing load scripts and scenarios
 Correlate unique test data to each user(ID) /session
 Pay strong attention to any detail of your load script and scenario, you have to think of
more things than you initially aware of….
 Use HTTP protocol if possible, until it appears that this becomes technically too complex,
then switch to GUI protocol
 Automate the correlation , to detect dynamic values in the script , so you can replace
dynamic data automaticallywith parameters
until it appears that this becomes technically too complex
until it appears that this becomes technically too complex
www.performancetestexpert.nl

35. This is what you should have just learned
A Productive way of load test development
Building effective tests !
….And you are now aware of (all) pieces:
(session) testdata,HTTP protocol,GUI level scripting,dynamic
data , parameterization,correlation,web 2.0, (a) sync requests,
websockets,think time,pacing,
assertions,transactions,validating scripts, configure scenario
(best practices), ….