Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
www.infopulse.com
Performance testing in 5 minutes
How to and tool comparison
1
www.infopulse.com
Who am I?
2
Oleksii Ostapov
• 11+ years in IT
• 5+ years as QA trainer
• 2+ years as test lead
• ISTQB c...
www.infopulse.com
Background
Bank
Risk management system
Telecom
Integration
Security Systems
Video management and access
www.infopulse.com
AGENDA
My goals
4
1
OVERVIEW
Give bread info about
performance testing
2
TELL SUCCESS
STORY
How I’ve don...
www.infopulse.com
Teach you how to test
Focus on rear cases you’d never use
Start holy war
01
02
03
NOT MY
GOALS
5
www.infopulse.com
PERFORMANCE TESTING BASED ON
6
Tools usually chosen by project
resource limits
RESOURCES
Technologies
La...
www.infopulse.com
PERFORMANCE DEPENDS ON
7
S H
NA
Client and server software,
frameworks, services, protocols
SOFTWARE
Pos...
www.infopulse.com
SUCCESS STORY 01
www.infopulse.com
LOCUST? 02
Free
Python
Cross Platform
Easy to start (1st test in 5 minutes)
Has web GUI
Distributed test...
www.infopulse.com
03AFTER
Tests done
Quality improved
Knowledge shared
www.infopulse.com
04
WHY NOT
<TOOL_NAME>?
www.infopulse.com
LET’S COMPARE! 05
Locust
Jmeter
Gatling
Tsung
Artillery
Yandex Tank
Load Runner
www.infopulse.com
LOCUST
Test sample
from locust import HttpLocust, TaskSet, task
class UserBehavior(TaskSet):
# runs one ...
www.infopulse.com
CUSTOMER FEEDBACK
Sugar plum gummi bears candy canes danish cake bear claw cake. Dessert bear claw powde...
www.infopulse.com
JMeter
Test Sample
www.infopulse.com
JMeter
Usage Sample
package Sample
import io.gatling.core.Predef._
import io.gatling.http.Predef._
www.infopulse.com
GATLING
Test Sample
package Sample
import io.gatling.core.Predef._
import io.gatling.http.Predef._
impor...
www.infopulse.com
GATLING
Usage Sample
www.infopulse.com
LET’S COMPARE
19
LOCUST
1. Python
2. Cross platform
3. 1 line command install
4. No record/playback
5. S...
www.infopulse.com
LET’S TEST
20
1
TEST PC
Load generator
2
NETWORK
Environment
3
TARGET SERVER
Under test
4
TEST DETAILS
S...
www.infopulse.com
TEST RESULTS 10 000
21
LOCUST JMETER GATLING
THROUGHPUT (RPS)1
TEST PC CPU, %
2
TEST PC RAM, GB
3
TEST P...
www.infopulse.com
???
22
www.infopulse.com
NEW TEST SETUP
23
Linear ramp
50 USERS / SEC
Total time
DURATION 2 MIN
For simultaneous load
500 USERS
www.infopulse.com
TEST RESULTS 500
24
LOCUST JMETER GATLING
THROUGHPUT (RPS)1
TEST PC CPU, %
2
TEST PC RAM, MB
3
TEST PC T...
www.infopulse.com 25
Performance testing much easier than it looks
CONCLUSIONS
1
www.infopulse.com 26
Test results analysis requires attention to make
tests better and got realistic software
characterist...
www.infopulse.com 27
All tools have their strong sides and can be
applied depends on test needs
TOOLS3
www.infopulse.com 28
Creating good performance test is like an art
OPTIMIZE 4
www.infopulse.com
THANK YOU!
QUESTIONS?
29https://github.com/Ypurek https://twitter.com/ypurek https://t.me/qamania
Upcoming SlideShare
Loading in …5
×

QA Fest 2019. Олексій Остапов. Тестування навантаження за 5 хв. Порівняння доступних засобів та корисні поради

20 views

Published on

Спочатку я хотів розповісти чергову success story - як ми “за 5 хвилин” протестували продуктивність сервісів в нашому проекті, але таких історій багато. Мені порадили зробити більше - саме так з’явилась ця доповідь. Будемо порівнювати можливості різних додатків для тестування навантаження та як іх використовувати.

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2019. Олексій Остапов. Тестування навантаження за 5 хв. Порівняння доступних засобів та корисні поради

  1. 1. www.infopulse.com Performance testing in 5 minutes How to and tool comparison 1
  2. 2. www.infopulse.com Who am I? 2 Oleksii Ostapov • 11+ years in IT • 5+ years as QA trainer • 2+ years as test lead • ISTQB certified • Test automation engineer
  3. 3. www.infopulse.com Background Bank Risk management system Telecom Integration Security Systems Video management and access
  4. 4. www.infopulse.com AGENDA My goals 4 1 OVERVIEW Give bread info about performance testing 2 TELL SUCCESS STORY How I’ve done performance testing 3 COMPARE TOOLS Test performance test tools and check their capabilities 4 TRY TO BE OBJECTIVE I have my favorite but open minded to discover new
  5. 5. www.infopulse.com Teach you how to test Focus on rear cases you’d never use Start holy war 01 02 03 NOT MY GOALS 5
  6. 6. www.infopulse.com PERFORMANCE TESTING BASED ON 6 Tools usually chosen by project resource limits RESOURCES Technologies Languages Tools SKILLS Benchmarking Load Stress Profiling GOALS T S R G Which technologies already used in project TECHNOLOGIES 1 2 4 3
  7. 7. www.infopulse.com PERFORMANCE DEPENDS ON 7 S H NA Client and server software, frameworks, services, protocols SOFTWARE Possibility to setup monitoring on server side ACCESS CPU, RAM, Adapters HARDWARE Topology, bandwidth, firewalls NETWORK
  8. 8. www.infopulse.com SUCCESS STORY 01
  9. 9. www.infopulse.com LOCUST? 02 Free Python Cross Platform Easy to start (1st test in 5 minutes) Has web GUI Distributed testing Battle tested *no locusts on this picture
  10. 10. www.infopulse.com 03AFTER Tests done Quality improved Knowledge shared
  11. 11. www.infopulse.com 04 WHY NOT <TOOL_NAME>?
  12. 12. www.infopulse.com LET’S COMPARE! 05 Locust Jmeter Gatling Tsung Artillery Yandex Tank Load Runner
  13. 13. www.infopulse.com LOCUST Test sample from locust import HttpLocust, TaskSet, task class UserBehavior(TaskSet): # runs one time for each user def on_start(self): self.client.get("/") @task(2) # chance to run 2/3 def posts(self): self.client.get("/posts") @task(1) # chance to run 1/3 def comment(self): data = { "postId": 1, "name": "my comment", "email": "test@user.test", "body": "Author is cool. Some text. Hello world!" } self.client.post("/comments", data) class WebsiteUser(HttpLocust): task_set = UserBehavior min_wait = 1000 max_wait = 2000
  14. 14. www.infopulse.com CUSTOMER FEEDBACK Sugar plum gummi bears candy canes danish cake bear claw cake. Dessert bear claw powder chocolate cake wafer fruitcake macaroon tart croissant. Pastry gummies marshmallow jelly beans gummi bears tart lollipop candy canes. Dessert bear Sugar plum gummi bears candy canes danish cake bear claw cake. Dessert bear claw powder chocolate cake wafer fruitcake macaroon tart croissant. Pastry gummies LOCUST Usage sample
  15. 15. www.infopulse.com JMeter Test Sample
  16. 16. www.infopulse.com JMeter Usage Sample package Sample import io.gatling.core.Predef._ import io.gatling.http.Predef._
  17. 17. www.infopulse.com GATLING Test Sample package Sample import io.gatling.core.Predef._ import io.gatling.http.Predef._ import scala.concurrent.duration._ class Simple2 extends Simulation { val httpProtocol = http .baseUrl("http://localhost:3000") val payload = """ { "postId": 1, "name": "my comment", "email": "test@user.habr", "body": "Author is cool. Some text. Hello world!" } """ val scn = scenario("sample 1") .exec(http("main page").get("/")).pause(1 second, 2 seconds) .forever(randomSwitch( 66.4 -> exec(http("posts").get("/posts")).pause(1 second, 2 seconds), 33.3 -> exec(http("comment").post("/comments").body(StringBody(payload)).asJson).pause(1 second, 2 seconds) )) setUp(scn.inject(rampUsers(10) during (10 seconds), nothingFor(100 seconds))) .pauses(normalPausesWithStdDevDuration(2 seconds)) .maxDuration(1 minute).protocols(httpProtocol) }
  18. 18. www.infopulse.com GATLING Usage Sample
  19. 19. www.infopulse.com LET’S COMPARE 19 LOCUST 1. Python 2. Cross platform 3. 1 line command install 4. No record/playback 5. Simple test in 5 minutes 6. Easy transaction tests 7. Only HTTP 8. Distributed testing 9. Live report, single format JMETER 1. Java 2. Cross platform 3. Unpack archive 4. Record/playback 5. Simple test in 10 minutes 6. Easy transaction tests 7. Supports various protocols 8. Distributed testing 9. Live report, various formats GATLING 1. Scala 2. Cross platform 3. Get Maven, plugins, archive 4. Record/playback 5. Simple test in 20 minutes 6. Quite easy transaction tests  7. Only HTTP 8. Distributed testing in Enterprise 9. Result report, single format
  20. 20. www.infopulse.com LET’S TEST 20 1 TEST PC Load generator 2 NETWORK Environment 3 TARGET SERVER Under test 4 TEST DETAILS Specification 1. CPU Core i7-7700 2.8 GHz 2. RAM 16 Gb 3. OS Windows 10 x64 Pro 4. Java 12 (Scala 2.13) 5. Python 3.6 1. Local network 2. Bandwidth 100 Mbps 1. Flask WS 2. Single “Hello World” endpoint 3. Docker python:3.7-alpine 4. RAM allocated 4 Gb 5. CPU Celeron J1900 2 GHz 1. Goal: check test tool behavior 2. 10 000 users 3. 100 sec linear ramp time 4. Duration 5 minutes 5. Request pause 1-2 sec 6. 1 GET request
  21. 21. www.infopulse.com TEST RESULTS 10 000 21 LOCUST JMETER GATLING THROUGHPUT (RPS)1 TEST PC CPU, % 2 TEST PC RAM, GB 3 TEST PC THREADS 4 TOTAL REQUESTS 5 SUCCESS RATE, % 6 AVERAGE RESPONSE TIME, SEC 7 350 250 190 20 3.6 5.4 13 80 40 0.5 2.3 1.2 14 10 039 77 92 000 415 600 341 400 100 14.9 * 17
  22. 22. www.infopulse.com ??? 22
  23. 23. www.infopulse.com NEW TEST SETUP 23 Linear ramp 50 USERS / SEC Total time DURATION 2 MIN For simultaneous load 500 USERS
  24. 24. www.infopulse.com TEST RESULTS 500 24 LOCUST JMETER GATLING THROUGHPUT (RPS)1 TEST PC CPU, % 2 TEST PC RAM, MB 3 TEST PC THREADS 4 TOTAL REQUESTS 5 SUCCESS RATE, % 6 AVERAGE RESPONSE TIME, MS 7 300 240 277 86 22 7 10 14 3 50 1 400 500 14 562 59 36 400 28 800 33 200 100 57 50
  25. 25. www.infopulse.com 25 Performance testing much easier than it looks CONCLUSIONS 1
  26. 26. www.infopulse.com 26 Test results analysis requires attention to make tests better and got realistic software characteristics TEST RESULT 2
  27. 27. www.infopulse.com 27 All tools have their strong sides and can be applied depends on test needs TOOLS3
  28. 28. www.infopulse.com 28 Creating good performance test is like an art OPTIMIZE 4
  29. 29. www.infopulse.com THANK YOU! QUESTIONS? 29https://github.com/Ypurek https://twitter.com/ypurek https://t.me/qamania

×