This document describes a performance automation solution using load testing scripts to continuously monitor application performance. The solution uses scripts to test functionality, availability, response times, and end-to-end workflows. Load testing engines run the scripts on a periodic schedule and store results. An alerting system analyzes results and sends alerts if response times exceed thresholds or tests fail to run. The system is containerized using Docker for scalability. Potential customers include project managers who need regression testing, monitoring of production applications, and emergency alerts about degradations or failures.

1. Performance Complete Automation success story
By Bruno AUDOUX - WAPSI - France

2. A - Functional Part

3. What isthe need?
• Is it possible to check coutinuous performance evolution of an
application ?
• Is it possible to garanty availability of an application ?
• How to ensure that End 2 End activity is ok ?
• What is response time evolution ?
• How to be alerted ?

4. What are theactual solution?
• Server and application Monitoring
• ➔ Garanty only server availability and application process running no more !
• Synthetic monitoring
• ➔Application availability, response time OK
• ➔ Only for Web application, not JSON / IOT services
• APM with Real User Monitoring
• ➔ Got Baseline and tendances, also history and alerts
• ➔ If services KO, no activity so no problem and nothing alert,
• ➔ RUM need javascript so no IOT supported

5. Begining ofsolution
• Only a loadtest script is able to check all of that :
• Functionality : OK
• Availability : Need periodic run
• Response time : OK
• End 2 End tests : OK
• Tendances : Need archives service with graphs dashboard
• Alerting : Need specific development

6. Scriptfunctionality
• Let’s have a exemple of script coming from an IOT like car onboard
computer and screen
• Update firmware dialog
• Connexion to server
• Sending inventory of Firmware version 1.0
• Requesting for package 2.0
• Downloading Package Version 2.0
• (instaling package Version 2.0)
• Sending inventory of Firmware version 2.0
• Leaving connexion

7. Howto do ?
Scripts
LoadTest Engine LoadTest Injector
LoadTest RepositoryAlert Dashboard
Scheduler

8. What to do?
Scripts
Functionality test
Get Response time
Send alert By API or Mail
• Step 1 : The Scripts
Functionality
Availability
Response time
End 2 End tests
Tendances
Alerting

9. What to do?
Scripts
LoadTest Engine LoadTest Injector
LoadTest Repository
Launch the runs
Connect the
application
Store the results
• Step 2 : The runs

10. What to do?
Scripts
LoadTest Engine LoadTest Injector
LoadTest Repository
Scheduler
Schedule type and
frequency
• Step 3 : Manage & Schedule
Functionality
Availability
Response time
End 2 End tests
Tendances
Alerting

11. What to do?
Scripts
LoadTest Engine LoadTest Injector
LoadTest RepositoryAlert Dashboard
Scheduler
Send Alerts to
dashboard or by mail
• Step 4 : Alerting
Send alert By API or Mail
Functionality
Availability
Response time
End 2 End tests
Tendances
Alerting

12. Which scriptto manage
• Main Script : Application test
• Test the application
• Get response time
• Get availability
• Result analyse Script and alert : Results Database
• Does the last test has passed from the last period (ex : 5 mins)
• Does response time are ok or need alerts ?
• Send alerts to Dashboard or Emails
• Maintenance script : in Results database
• Clean result over 1 month
Step 1 test
Step 2 Alert
Step 3
Maintenance

13. B - Technical Part
Now that everybody understoud what is it, let see
the technical part !

14. Howto do ?
Scripts
LoadTest Engine
LoadTest Injector
LoadTest Repository
Alert Dashboard
Scheduler
Neoload
Controler
Neoload
Injector
Neoload
Web
OTHER

15. Jenkins PiPELINE commandline
• sh '/home/neoload/bin/NeoLoadCmd
• -project ${WORKSPACE}/NeoloadProject.nlp
• -testResultName « CountinuousPerformance (build ${BUILD_NUMBER})"
• -noGUI
• -launch CountinuousPerformance_Scenario
• -loadGenerators "${WORKSPACE}/loadgenerators.yaml"
• -nlweb -nlwebToken xxxyyyxxxyyyy
• -nlwebAPIURL "http://docker-proto:8080"
• -leaseServer nlweb -leaseLicense 5:1'

16. The advantages ofDOCKER
• Periodic test use real advantages of DOCKER & MicroService method
• Example of a run :
Neoload
Controler
Neoload
Injector
Temporary Microservices
Neoload
Web
Permanent Microservices

17. Dashboard, storage antendances

18. Alookinside thescripts

19. Alert
• Which kind of alert
• Performance degradation alert ➔ Need SLA to compare
• Error alert (error 500 or 401)
• No periodic test alert ➔ test also the test tool ( check if last test is less than 5 mins)
• Nature of alert
• A single mail
• Message to Slack by Webhook
• Dashboard status evolution (green, orange, red in aTrello column)
• Other kind of API tool

20. C - Business Part
For shure you understoud how does it work, so
let’s talk about business

21. Who couldbeinterested ?
• Agile project manager during developement phases
• Help for Regression Testing, capability and sizing
• Project Manager in production (Agile or not)
• Application monitoring with emergency alert
• Network / infrastructure responsable
• Periodic or erratic response time degradation
• Emergency alert of network failure
• SaaS/PaaS manager
• KPI of availability
• Baseline garanty

22. The end
• Any question ?