Successfully reported this slideshow.
Upcoming SlideShare
×

# Performance tests with Gatling (extended)

357 views

Published on

Theoretical and practical introduction to performance tests topic.

Published in: Software
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

### Performance tests with Gatling (extended)

1. 1. Performance tests with Gatling Andrzej Ludwikowski
2. 2. About me ➔ ➔ aludwikowski.blogspot.com ➔ github.com/aludwiko ➔ @aludwikowski
3. 3. Performance tests - why so hard? Simulate production as close as possible: ● hardware ○ CPU, RAM, storage, ... ● software ○ OS, Virtualization, DBs, … ● load ● isolation
4. 4. Performance tests - why so hard? Necessary to have: ● infrastructure ● monitoring ● logging
5. 5. Performance tests - why so hard? ● Your performance intuition is wrong! 1. collect the data (monitoring, logging, profiling) 2. find the bottleneck (based on data) 3. fix the bottleneck 4. collect the data and check the assumptions 5. go to 1.
6. 6. Performance tests - why so hard? Lies, damned lies, and statistics: ● arithmetic mean = 2.9 ● median = 1 ● standard deviation = 6 (only for normal distribution)
7. 7. Performance tests - why so hard? ● Anscombe's quartet http://bravenewgeek.com/tag/coordinated-omission/ Property Value Mean of x 9 Sample variance of x 11 Mean of y 7.50 Sample variance of y 4.125 Correlation between x and y 0.816 Linear regression line y = 3 + 0.5x Coefficient of determination of the linear regression 0.67
8. 8. Performance tests - why so hard? Lies, damned lies, and statistics: ● arithmetic mean = 2.9 ● median = 1 ● standard deviation = 6 (only for normal distribution) Use: ● percentiles ○ 50th = 1 ○ 70th = 1 ○ 90th = 2.9 ○ 95th = 11.45 ○ 99th = 18.29 Check percentiles implementation!
9. 9. Performance tests - why so hard? ● Coordinated omission problem by Gil Tene http://bravenewgeek.com/tag/coordinated-omission/
10. 10. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0)
11. 11. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 31 50 th 1 70 th 1 90 th 1 95 th 1 99 th 12.89 99,9 th 28.30 99,99 th 29.82 avg 1.93
12. 12. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 10001 50 th 1 70 th 1 90 th 1 95 th 1 99 th 1 99,9 th 1.15 99,99 th 28.01 avg 1.03
13. 13. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0)
14. 14. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 60 50 th 1 70 th 12.6 90 th 24.3 95 th 27.2 99 th 29.45 99,9 th 29.95 99,99 th 29.99 avg 8.25
15. 15. Coordinated omission problem
16. 16. Coordinated omission problem total time 60 s max 30 s 99th 1 s
17. 17. Coordinated omission problem total time 60 s max 30 s 99th 1 s time in % for max 50%
18. 18. Coordinated omission problem total time 60 s max 30 s 99th 1 s time in % for max 50% expected time in % for 99th 50% - 1% = 49%
19. 19. Coordinated omission problem total time 60 s max 30 s 99th 1 s time in % for max 50% expected time in % for 99th 50% - 1% = 49% real time for 99th 60 s * 49% = 29.4 s
20. 20. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/
21. 21. Performance tests - why so hard? ● Coordinated omission vs Gatling http://bravenewgeek.com/tag/coordinated-omission/
22. 22. Performance tests - why so hard? ● Tests…
23. 23. Performance tests - why so hard? ● Tests…
24. 24. Gatling
25. 25. Gatling
26. 26. Why Gatling? ● non-blocking, asynchronous stack (scala, akka, netty) ● scala !!!111oneoneone (maven, sbt support) ● DSL ● recorder ● math is good ● reports ● integration & performance tests
27. 27. Gatling DSL class BasicSimulation extends Simulation { val httpConf = http .baseURL("http://computer-database.gatling.io") .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") .acceptEncodingHeader("gzip, deflate") .acceptLanguageHeader("en-US,en;q=0.5") .userAgentHeader("Mozilla/5.0 (Macintosh; X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0") val firstScenario = scenario("First Scenario Name") .exec(http("Request name").get("/")) .pause(7) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
28. 28. Gatling DSL class FirstScenarioV2 extends MySimulation { val firstScenario = scenario("First Scenario Name") .exec(http("Go to root page").get("/")) .pause(7 seconds) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
29. 29. Gatling DSL import io.gatling.core.Predef._ import io.gatling.http.Predef._ class FirstScenarioV2 extends MySimulation { val firstScenario = scenario("First Scenario Name") .exec(http("Go to root page").get("/")) .pause(7 seconds) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
30. 30. Gatling DSL class ComplexScenario extends MySimulation { val complexScenario = scenario("Complex demo scenario") .exec(http("request_1").get("/")).pause(7) .exec(http("request_2").get("/computers?f=macbook")).pause(2) .exec(http("request_3").get("/computers/6")).pause(3) .exec(http("request_4").get("/")).pause(2) .exec(http("request_5").get("/computers?p=1")).pause(670 milliseconds) .exec(http("request_6").get("/computers/new")).pause(1) .exec(http("request_7") .post("/computers") .formParam("name", "MyComputer").formParam("introduced", "2012-05-30").formParam("company", "37")) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
31. 31. Gatling DSL class ComplexScenarioV2 extends MySimulation { val complexScenario = scenario("Complex demo scenario") .exec(goToRootPage).pause(7) .exec(searchFor("macbook")).pause(2) .exec(positionAt(6)).pause(3) .exec(goToRootPage).pause(2) .exec(goToPage(1)).pause(670 milliseconds) .exec(openNewComputerForm).pause(1) .exec(addNewComputer) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
32. 32. Gatling DSL class ComplexScenarioV3 extends MySimulation { val search = exec(goToRootPage).pause(7) .exec(searchFor("macbook")).pause(2) .exec(positionAt(6)).pause(3) val addComputer = exec(goToRootPage).pause(2) .exec(goToPage(1)).pause(670 milliseconds) .exec(openNewComputerForm).pause(1) .exec(addNewComputer) val complexScenario = scenario("Complex demo scenario").exec(search, addComputer) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
33. 33. Gatling DSL - checks scenario("DSL demo") .exec(http("go to page") .get("/computers") .check(status.is(200)) .check(status.in(200 to 210))
34. 34. Gatling DSL - checks scenario("DSL demo") .exec(http("go to page") .get("/computers") .check(regex("computers") .find(1) .exists) https://www.pinterest.com/pin/491666484294006138/
35. 35. Gatling DSL - checks scenario("First Scenario Name") .exec(http("Request name") .get("/computers") .check(jsonPath("\$..foo.bar[2].baz").ofType[String].notExists) .check(xpath("//input[@id='text1']/@value").is("test")) .check(css("...").transform(_.split('|').toSeq).is(Seq("1", "2")))
36. 36. Gatling DSL - checks scenario("DSL demo") .exec(http("Authorize") .get("/auth") .check(regex("token").find(1).exists .saveAs("authorizationToken"))) .exec(http("Authorized resource") .get("/authorized_resource?token=\${authorizationToken}"))
37. 37. Gatling DSL - looping repeat(5, "i") { exec(goToPage("\${i}".toInt)) .pause(1) } ● repeat ● foreach ● during ● asLongAs ● forever https://blog.hubspot.com/blog/tabid/6307/bid/32019/Why-Every-Marketer-Needs-Closed-Loop-Reporting.aspx#sm.0005lrqj811waf3ntmn1cul3881gr
38. 38. Gatling DSL - polling exec( polling .every(10 seconds) .exec(searchFor("thinkpad")) ) http://www.firmus-solutions.com/terms-conditions.html
39. 39. Gatling DSL - conditions doIf(session => session("user").as[String].startsWith("admin")) { exec(goToAdminPage) } ● doIf ● doIfEquals ● doIfOrElse ● doSwitch ● doSwitchOrElse ● randomSwitch https://en.wikipedia.org/wiki/Decision_tree
40. 40. Gatling DSL - error management exec(sendMoney) .tryMax(10){ exec(checkIfMoneyReceived) } ● tryMax ● exitBlockOnFail ● exitHereIfFailed Alice Bob Kafka
41. 41. Gatling DSL - setup setUp(myScenario .inject( nothingFor(4 seconds), atOnceUsers(10), rampUsers(10) over (5 seconds)) .protocols(httpConf)) .maxDuration(10 minutes) ● constantUsersPerSec ● rampUsersPerSec ● splitUsers ● heavisideUsers
42. 42. Gatling DSL - setup setUp(myScenario .inject(atOnceUsers(10)) .protocols(httpConf)) .assertions( global.responseTime.max.lt(50), global.failedRequests.percent.is(0) ) http://englishthroughlaxas.blogspot.com/2015/07/531-24-expression-of-assertion-emphasis.html
43. 43. Gatling DSL - setup setUp(myScenario .inject(atOnceUsers(10)) .protocols(httpConf)) .throttle( reachRps(100) in (30 second), holdFor(1 minute), jumpToRps(50), holdFor(2 hours) )
44. 44. Gatling DSL - feeders val companies = List("apple", "lenovo", "hp") val feeder = Iterator.continually( Map("company" -> companies(Random.nextInt(companies.size)))) val searching = scenario("Searching") .feed(feeder) .exec(searchFor("\${company}")) ● RecordSeqFeederBuilder ● CSV ● JSON ● JDBC ● Sitemap ● Redis ● … http://favim.com/orig/201104/23/Favim.com-22725.jpg