Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PAC 2019 virtual Bruno Audoux

Performance automation, we can do it!

  • Be the first to comment

  • Be the first to like this

PAC 2019 virtual Bruno Audoux

  1. 1. Performance Complete Automation success story By Bruno AUDOUX - WAPSI - France
  2. 2. A - Functional Part
  3. 3. What isthe need? • Is it possible to check coutinuous performance evolution of an application ? • Is it possible to garanty availability of an application ? • How to ensure that End 2 End activity is ok ? • What is response time evolution ? • How to be alerted ?
  4. 4. What are theactual solution? • Server and application Monitoring • ➔ Garanty only server availability and application process running no more ! • Synthetic monitoring • ➔Application availability, response time OK • ➔ Only for Web application, not JSON / IOT services • APM with Real User Monitoring • ➔ Got Baseline and tendances, also history and alerts • ➔ If services KO, no activity so no problem and nothing alert, • ➔ RUM need javascript so no IOT supported
  5. 5. Begining ofsolution • Only a loadtest script is able to check all of that : • Functionality : OK • Availability : Need periodic run • Response time : OK • End 2 End tests : OK • Tendances : Need archives service with graphs dashboard • Alerting : Need specific development
  6. 6. Scriptfunctionality • Let’s have a exemple of script coming from an IOT like car onboard computer and screen • Update firmware dialog • Connexion to server • Sending inventory of Firmware version 1.0 • Requesting for package 2.0 • Downloading Package Version 2.0 • (instaling package Version 2.0) • Sending inventory of Firmware version 2.0 • Leaving connexion
  7. 7. Howto do ? Scripts LoadTest Engine LoadTest Injector LoadTest RepositoryAlert Dashboard Scheduler
  8. 8. What to do? Scripts Functionality test Get Response time Send alert By API or Mail • Step 1 : The Scripts Functionality Availability Response time End 2 End tests Tendances Alerting
  9. 9. What to do? Scripts LoadTest Engine LoadTest Injector LoadTest Repository Launch the runs Connect the application Store the results • Step 2 : The runs
  10. 10. What to do? Scripts LoadTest Engine LoadTest Injector LoadTest Repository Scheduler Schedule type and frequency • Step 3 : Manage & Schedule Functionality Availability Response time End 2 End tests Tendances Alerting
  11. 11. What to do? Scripts LoadTest Engine LoadTest Injector LoadTest RepositoryAlert Dashboard Scheduler Send Alerts to dashboard or by mail • Step 4 : Alerting Send alert By API or Mail Functionality Availability Response time End 2 End tests Tendances Alerting
  12. 12. Which scriptto manage • Main Script : Application test • Test the application • Get response time • Get availability • Result analyse Script and alert : Results Database • Does the last test has passed from the last period (ex : 5 mins) • Does response time are ok or need alerts ? • Send alerts to Dashboard or Emails • Maintenance script : in Results database • Clean result over 1 month Step 1 test Step 2 Alert Step 3 Maintenance
  13. 13. B - Technical Part Now that everybody understoud what is it, let see the technical part !
  14. 14. Howto do ? Scripts LoadTest Engine LoadTest Injector LoadTest Repository Alert Dashboard Scheduler Neoload Controler Neoload Injector Neoload Web OTHER
  15. 15. Jenkins PiPELINE commandline • sh '/home/neoload/bin/NeoLoadCmd • -project ${WORKSPACE}/NeoloadProject.nlp • -testResultName « CountinuousPerformance (build ${BUILD_NUMBER})" • -noGUI • -launch CountinuousPerformance_Scenario • -loadGenerators "${WORKSPACE}/loadgenerators.yaml" • -nlweb -nlwebToken xxxyyyxxxyyyy • -nlwebAPIURL "http://docker-proto:8080" • -leaseServer nlweb -leaseLicense 5:1'
  16. 16. The advantages ofDOCKER • Periodic test use real advantages of DOCKER & MicroService method • Example of a run : Neoload Controler Neoload Injector Temporary Microservices Neoload Web Permanent Microservices
  17. 17. Dashboard, storage antendances
  18. 18. Alookinside thescripts
  19. 19. Alert • Which kind of alert • Performance degradation alert ➔ Need SLA to compare • Error alert (error 500 or 401) • No periodic test alert ➔ test also the test tool ( check if last test is less than 5 mins) • Nature of alert • A single mail • Message to Slack by Webhook • Dashboard status evolution (green, orange, red in aTrello column) • Other kind of API tool
  20. 20. C - Business Part For shure you understoud how does it work, so let’s talk about business
  21. 21. Who couldbeinterested ? • Agile project manager during developement phases • Help for Regression Testing, capability and sizing • Project Manager in production (Agile or not) • Application monitoring with emergency alert • Network / infrastructure responsable • Periodic or erratic response time degradation • Emergency alert of network failure • SaaS/PaaS manager • KPI of availability • Baseline garanty
  22. 22. The end • Any question ?

×