Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Last Minute Performance Testing
t WITH PASSION TO QUALITY
Pavlo Vedilin
QA CONFERENCE #1 IN UKRAINE, KYIV 2018
PROJECTS
A B C
E FDCross-company
structure that serves as
a center of excellence
for delivery teams
G H
L NK
J
GOOD NEWS, EVERYONE!
Dear QMO
We need help with Load Testing for our project. It’s one-time activity that should be run
du...
LOAD TESTING?
Virtual User 1
Virtual User 2
Virtual User N
…
SO WHAT IS THE PLAN?
The performance testing approach consists of the following activities*:
1. Identify the Test Environm...
WEEK 1
PM:
• Client is a e-learning developer and provider
• Client`s platform consists of several servers with different roles
• Sho...
WHAT?
WHAT?
WHAT?
WHAT NOW?
ANY OTHER TOOLS?
ANY OTHER TOOLS?
ANY OTHER TOOLS?
IT WORKS ON MY MACHINE!
WHAT ABOUT CLIENT ENVIRONMENT?
CLIENT CALL
US:
– WE FOUND A TOOL THAT WORKS
– WE DESIGNED THE TEST
– WORKING ON MONITORING OF THE APP
– WORKING ON TEST S...
WEEK 2
CLIENT:PM:
SO WHATS WRONG WITH
CLIENT ENVIRONMENT?
STICKY SESSIONS
1
2
3
Hi!
Hey 1
1 1 Tsup?
STICKY SESSIONS
1
2
3
Hi!
Hey 1
Tsup?
?
• TSUNG SUPPORTS COOKIES
• …BUT ONLY IN HTTP MODULE
HTTP Module
BOSH Module
CALL
FOR
HELP
TIME TO…
CLIENT CALL
US:
– IT’S DEADLINE…
– … AND WE NEED TO ADD COOKIES SUPPORT TO TSUNG
CLIENT:
– YOU ARE WASTING TIME!
– DO IT W...
WEEK 3
CLIENT:PM:
THREAD 2
 CREATE NEW TOOL WITH JS
HELP NEEDED!
THREAD 1
 ADD COOKIE SUPPORT ?
PROVE ME THAT TSUNG WORKS!
WHAT ELSE CAN GO WRONG?
PROBLEM SOLVED!
13 MINUTES INVISIBLE WALL
CLIENT CALL
1
2
3
Hi!
Hey 1
1 Tsup?
1
WHAT HAPPENED?
WEEK 4
CLIENT:PM:
IS IT WORKING YET?
SO WHAT WAS THE PLAN?
The performance testing approach consists of the following activities*:
1. Identify the Test Environ...
ONE
MONTH
LATER
ME:
RESULTS
 GENERALLY PESSIMISTIC TEST RESULTS
 CLIENT DECIDES TO COMPLETELY CHANGE ARCHITECTURE
 … AND TO USE THIS ALTERE...
QUESTIONS?
LESSONS LEARNED
 DON’T LEAVE ‘SIMPLE’ TASKS FOR LAST
 …BUT IF YOU DID…
 DON’T PANIC!
 EVEN IF EVERYBODY AROUND YOU ARE...
THANK YOU
Тема доклада
Тема доклада
Тема доклада
KYIV 2018
WITH PASSION TO QUALITY
QA CONFERENCE #1 IN UKRAINE
QA Fest 2018. Павел Ведилин. Нагрузочное тестирование в последний момент и что из этого получилось
Upcoming SlideShare
Loading in …5
×

QA Fest 2018. Павел Ведилин. Нагрузочное тестирование в последний момент и что из этого получилось

27 views

Published on

Я расскажу про один из проектов по тестированию производительности в котором я принимал непосредственное участие. Это история о том, как все на проекте может пойти не так; о необычных технологиях, инструментах и подходах; про паникующих клиентов и руководство. И о том, как с этим справиться и не сойти с ума.

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2018. Павел Ведилин. Нагрузочное тестирование в последний момент и что из этого получилось

  1. 1. Last Minute Performance Testing t WITH PASSION TO QUALITY Pavlo Vedilin QA CONFERENCE #1 IN UKRAINE, KYIV 2018
  2. 2. PROJECTS A B C E FDCross-company structure that serves as a center of excellence for delivery teams G H L NK J
  3. 3. GOOD NEWS, EVERYONE! Dear QMO We need help with Load Testing for our project. It’s one-time activity that should be run during the next 2 weeks. We don’t have QC there, only DevOps who don’t have experience with Load tests. Best Regards Name Here
  4. 4. LOAD TESTING? Virtual User 1 Virtual User 2 Virtual User N …
  5. 5. SO WHAT IS THE PLAN? The performance testing approach consists of the following activities*: 1. Identify the Test Environment. 2. Identify Performance Acceptance Criteria. 3. Plan and Design Tests. 4. Configure the Test Environment. 5. Implement the Test Design. 6. Execute the Test. 7. Analyze Results, Report, and Retest. * According to Microsoft “Performance Testing Guidance for Web Applications”:
  6. 6. WEEK 1 PM:
  7. 7. • Client is a e-learning developer and provider • Client`s platform consists of several servers with different roles • Short-term DevOps contract • Goal is to write scripts for quick client infrastructure set-up • Contract includes an agreement to check if environment will not fail under the load KICK-OFF MEETING 20 000 USERS VAGRANT CHEF PUNJAB OPENFIRE JABBER XMPP WEBSOCKETS BOSH PROTOCOL DEADLINE IN TWO WEEKS!
  8. 8. WHAT?
  9. 9. WHAT?
  10. 10. WHAT?
  11. 11. WHAT NOW?
  12. 12. ANY OTHER TOOLS?
  13. 13. ANY OTHER TOOLS?
  14. 14. ANY OTHER TOOLS?
  15. 15. IT WORKS ON MY MACHINE!
  16. 16. WHAT ABOUT CLIENT ENVIRONMENT?
  17. 17. CLIENT CALL US: – WE FOUND A TOOL THAT WORKS – WE DESIGNED THE TEST – WORKING ON MONITORING OF THE APP – WORKING ON TEST SCRIPT IMPLEMENTATION – … BUT FOR NOW IT DOESN’T RUN ON YOUR ENVIRONMENT CLIENT: – THIS IS NOT WHAT I EXPECTED! – IT WILL NOT WORK! – DO IT WITH JAVASCRIPT!
  18. 18. WEEK 2 CLIENT:PM:
  19. 19. SO WHATS WRONG WITH CLIENT ENVIRONMENT?
  20. 20. STICKY SESSIONS 1 2 3 Hi! Hey 1 1 1 Tsup?
  21. 21. STICKY SESSIONS 1 2 3 Hi! Hey 1 Tsup? ?
  22. 22. • TSUNG SUPPORTS COOKIES • …BUT ONLY IN HTTP MODULE
  23. 23. HTTP Module BOSH Module
  24. 24. CALL FOR HELP TIME TO…
  25. 25. CLIENT CALL US: – IT’S DEADLINE… – … AND WE NEED TO ADD COOKIES SUPPORT TO TSUNG CLIENT: – YOU ARE WASTING TIME! – DO IT WITH JAVASCRIPT! – … WE’LL GIVE YOU ADDITIONAL TIME FOR THAT
  26. 26. WEEK 3 CLIENT:PM:
  27. 27. THREAD 2  CREATE NEW TOOL WITH JS HELP NEEDED! THREAD 1  ADD COOKIE SUPPORT ?
  28. 28. PROVE ME THAT TSUNG WORKS!
  29. 29. WHAT ELSE CAN GO WRONG? PROBLEM SOLVED!
  30. 30. 13 MINUTES INVISIBLE WALL
  31. 31. CLIENT CALL 1 2 3 Hi! Hey 1 1 Tsup? 1
  32. 32. WHAT HAPPENED?
  33. 33. WEEK 4 CLIENT:PM:
  34. 34. IS IT WORKING YET?
  35. 35. SO WHAT WAS THE PLAN? The performance testing approach consists of the following activities*: 1. Identify the Test Environment. 2. Identify Performance Acceptance Criteria. 3. Plan and Design Tests. 4. Configure the Test Environment. 5. Implement the Test Design. 6. Execute the Test. 7. Analyze Results, Report, and Retest. * According to Microsoft “Performance Testing Guidance for Web Applications”:
  36. 36. ONE MONTH LATER ME:
  37. 37. RESULTS  GENERALLY PESSIMISTIC TEST RESULTS  CLIENT DECIDES TO COMPLETELY CHANGE ARCHITECTURE  … AND TO USE THIS ALTERED TOOL FOR HEALTH CHECKS  CLIENT CONTINUED COOPERATION WITH SOFTSERVE
  38. 38. QUESTIONS?
  39. 39. LESSONS LEARNED  DON’T LEAVE ‘SIMPLE’ TASKS FOR LAST  …BUT IF YOU DID…  DON’T PANIC!  EVEN IF EVERYBODY AROUND YOU ARE IN PANIC  COMMUNICATION IS KEY  KEEP STAKEHOLDERS UPDATED  IF STUCK, CALL FOR HELP  LOG YOUR WORK  … AND ASK FOR DETAILED ARCHITECTURE
  40. 40. THANK YOU
  41. 41. Тема доклада Тема доклада Тема доклада KYIV 2018 WITH PASSION TO QUALITY QA CONFERENCE #1 IN UKRAINE

×