Я расскажу про один из проектов по тестированию производительности в котором я принимал непосредственное участие. Это история о том, как все на проекте может пойти не так; о необычных технологиях, инструментах и подходах; про паникующих клиентов и руководство. И о том, как с этим справиться и не сойти с ума.
QA Fest 2018. Павел Ведилин. Нагрузочное тестирование в последний момент и что из этого получилось
1. Last Minute Performance Testing
t WITH PASSION TO QUALITY
Pavlo Vedilin
QA CONFERENCE #1 IN UKRAINE, KYIV 2018
2. PROJECTS
A B C
E FDCross-company
structure that serves as
a center of excellence
for delivery teams
G H
L NK
J
3. GOOD NEWS, EVERYONE!
Dear QMO
We need help with Load Testing for our project. It’s one-time activity that should be run
during the next 2 weeks. We don’t have QC there, only DevOps who don’t have experience
with Load tests.
Best Regards
Name Here
5. SO WHAT IS THE PLAN?
The performance testing approach consists of the following activities*:
1. Identify the Test Environment.
2. Identify Performance Acceptance Criteria.
3. Plan and Design Tests.
4. Configure the Test Environment.
5. Implement the Test Design.
6. Execute the Test.
7. Analyze Results, Report, and Retest.
* According to Microsoft “Performance Testing Guidance for Web Applications”:
7. • Client is a e-learning developer and provider
• Client`s platform consists of several servers with different roles
• Short-term DevOps contract
• Goal is to write scripts for quick client infrastructure set-up
• Contract includes an agreement to check if environment will not fail under the load
KICK-OFF MEETING
20 000 USERS
VAGRANT
CHEF
PUNJAB
OPENFIRE
JABBER
XMPP
WEBSOCKETS
BOSH PROTOCOL
DEADLINE IN TWO WEEKS!
17. CLIENT CALL
US:
– WE FOUND A TOOL THAT WORKS
– WE DESIGNED THE TEST
– WORKING ON MONITORING OF THE APP
– WORKING ON TEST SCRIPT IMPLEMENTATION
– … BUT FOR NOW IT DOESN’T RUN ON YOUR ENVIRONMENT
CLIENT:
– THIS IS NOT WHAT I EXPECTED!
– IT WILL NOT WORK!
– DO IT WITH JAVASCRIPT!
25. CLIENT CALL
US:
– IT’S DEADLINE…
– … AND WE NEED TO ADD COOKIES SUPPORT TO TSUNG
CLIENT:
– YOU ARE WASTING TIME!
– DO IT WITH JAVASCRIPT!
– … WE’LL GIVE YOU ADDITIONAL TIME FOR THAT
36. SO WHAT WAS THE PLAN?
The performance testing approach consists of the following activities*:
1. Identify the Test Environment.
2. Identify Performance Acceptance Criteria.
3. Plan and Design Tests.
4. Configure the Test Environment.
5. Implement the Test Design.
6. Execute the Test.
7. Analyze Results, Report, and Retest.
* According to Microsoft “Performance Testing Guidance for Web Applications”:
38. RESULTS
GENERALLY PESSIMISTIC TEST RESULTS
CLIENT DECIDES TO COMPLETELY CHANGE ARCHITECTURE
… AND TO USE THIS ALTERED TOOL FOR HEALTH CHECKS
CLIENT CONTINUED COOPERATION WITH SOFTSERVE
40. LESSONS LEARNED
DON’T LEAVE ‘SIMPLE’ TASKS FOR LAST
…BUT IF YOU DID…
DON’T PANIC!
EVEN IF EVERYBODY AROUND YOU ARE IN PANIC
COMMUNICATION IS KEY
KEEP STAKEHOLDERS UPDATED
IF STUCK, CALL FOR HELP
LOG YOUR WORK
… AND ASK FOR DETAILED ARCHITECTURE