Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Performance tests with Gatling

158 views

Published on

Slides from Confitura 2017: https://2017.confitura.pl/presentations

Published in: Software
  • Be the first to comment

  • Be the first to like this

Performance tests with Gatling

  1. 1. Performance tests with Gatling Andrzej Ludwikowski
  2. 2. About me ➔aludwikowski.blogspot.com ➔github.com/aludwiko ➔@aludwikowski
  3. 3. Performance tests - why so hard? Simulate production as close as possible: hardware CPU, RAM, storage, ... software OS, Virtualization, DBs, … load isolation
  4. 4. Performance tests - why so hard? Necessary to have: infrastructure monitoring logging
  5. 5. Performance tests - why so hard? Your performance intuition is wrong!
  6. 6. Performance tests - why so hard? Lies, damned lies, and statistics: arithmetic mean = 2.9 median = 1 standard deviation = 6 (only for normal distribution) Use: percentiles 50 = 1 70 = 1 90 = 2.9 95 = 11.45 99 = 18.29 Check percentiles implementation!
  7. 7. Performance tests - why so hard? ● Coordinated omission problem by Gil Tene http://bravenewgeek.com/tag/coordinated-omission/
  8. 8. Performance tests - why so hard? http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0)
  9. 9. Performance tests - why so hard? http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 31 50 % 1 70 % 1 90 % 1 95 % 1 99 % 12,89 99,9 % 28,30 99,99 % 29,82 avg 1,93
  10. 10. Performance tests - why so hard? http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 10001 50 % 1 70 % 1 90 % 1 95 % 1 99 % 1 99,9 % 1,15 99,99 % 28,01 avg 1,03
  11. 11. Performance tests - why so hard? http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0)
  12. 12. Performance tests - why so hard? http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 60 50 % 1 70 % 12,6 90 % 24,3 95 % 27,2 99 % 29,45 99,9 % 29,95 99,99 % 29,99 avg 8,25
  13. 13. Performance tests - why so hard?
  14. 14. Performance tests - why so hard? total time 60 s max 30 s 99% 1 s
  15. 15. Performance tests - why so hard? total time 60 s max 30 s 99% 1 s time in % for max 50%
  16. 16. Performance tests - why so hard? total time 60 s max 30 s 99% 1 s time in % for max 50% expected time in % for 99% 50% - 1% = 49%
  17. 17. Performance tests - why so hard? total time 60 s max 30 s 99% 1 s time in % for max 50% expected time in % for 99% 50% - 1% = 49% real time for 99% 60 s * 49% = 29,4 s
  18. 18. Performance tests - why so hard? http://bravenewgeek.com/tag/coordinated-omission/
  19. 19. Performance tests - why so hard? Tests…
  20. 20. Performance tests - why so hard? Tests…
  21. 21. Gatling
  22. 22. Gatling
  23. 23. Why Gatling? non-blocking, asynchronous stack (scala, akka, netty) scala !!!111oneoneone (maven, sbt support) DSL recorder math is good reports integration & performance tests
  24. 24. Gatling DSL class BasicSimulation extends Simulation { val httpConf = http .baseURL("http://computer-database.gatling.io") .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") .acceptEncodingHeader("gzip, deflate") .acceptLanguageHeader("en-US,en;q=0.5") .userAgentHeader("Mozilla/5.0 (Macintosh; X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0") val firstScenario = scenario("First Scenario Name") .exec(http("Request name").get("/")) .pause(7) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  25. 25. Gatling DSL class FirstScenarioV2 extends MySimulation { val firstScenario = scenario("First Scenario Name") .exec(http("Go to root page").get("/")) .pause(7 seconds) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  26. 26. Gatling DSL import io.gatling.core.Predef._ import io.gatling.http.Predef._ class FirstScenarioV2 extends MySimulation { val firstScenario = scenario("First Scenario Name") .exec(http("Go to root page").get("/")) .pause(7 seconds) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  27. 27. Gatling DSL class ComplexScenario extends MySimulation { val complexScenario = scenario("Complex demo scenario") .exec(http("request_1").get("/")).pause(7) .exec(http("request_2").get("/computers?f=macbook")).pause(2) .exec(http("request_3").get("/computers/6")).pause(3) .exec(http("request_4").get("/")).pause(2) .exec(http("request_5").get("/computers?p=1")).pause(670 milliseconds) .exec(http("request_6").get("/computers/new")).pause(1) .exec(http("request_7") .post("/computers") .formParam("name", "MyComputer").formParam("introduced", "2012-05-30").formParam("company", "37")) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  28. 28. Gatling DSL class ComplexScenarioV2 extends MySimulation { val complexScenario = scenario("Complex demo scenario") .exec(goToRootPage).pause(7) .exec(searchFor("macbook")).pause(2) .exec(positionAt(6)).pause(3) .exec(goToRootPage).pause(2) .exec(goToPage(1)).pause(670 milliseconds) .exec(openNewComputerForm).pause(1) .exec(addNewComputer) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  29. 29. Gatling DSL class ComplexScenarioV3 extends MySimulation { val search = exec(goToRootPage).pause(7) .exec(searchFor("macbook")).pause(2) .exec(positionAt(6)).pause(3) val addComputer = exec(goToRootPage).pause(2) .exec(goToPage(1)).pause(670 milliseconds) .exec(openNewComputerForm).pause(1) .exec(addNewComputer) val complexScenario = scenario("Complex demo scenario").exec(search, addComputer) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  30. 30. Gatling DSL - checks scenario("DSL demo") .exec(http("go to page") .get("/computers") .check(status.is(200)) .check(status.in(200 to 210))
  31. 31. Gatling DSL - checks scenario("DSL demo") .exec(http("go to page") .get("/computers") .check(regex("computers") .find(1) .exists) https://www.pinterest.com/pin/491666484294006138/
  32. 32. Gatling DSL - checks scenario("First Scenario Name") .exec(http("Request name") .get("/computers") .check(jsonPath("$..foo.bar[2].baz").ofType[String].notExists) .check(xpath("//input[@id='text1']/@value").is("test")) .check(css("...").transform(_.split('|').toSeq).is(Seq("1", "2")))
  33. 33. Gatling DSL - checks scenario("DSL demo") .exec(http("Authorize") .get("/auth") .check(regex("token").find(1).exists .saveAs("authorizationToken"))) .exec(http("Authorized resource") .get("/authorized_resource?token=${authorizationToken}"))
  34. 34. Gatling DSL - looping repeat(5, "i") { exec(goToPage("${i}".toInt)) .pause(1) } repeat foreach during asLongAs forever https://blog.hubspot.com/blog/tabid/6307/bid/32019/Why-Every-Marketer-Needs-Closed-Loop-Reporting.aspx#sm.0005lrqj811waf3ntmn1cul3881gr
  35. 35. Gatling DSL - polling exec( polling .every(10 seconds) .exec(searchFor("thinkpad")) ) http://www.firmus-solutions.com/terms-conditions.html
  36. 36. Gatling DSL - conditions doIf(session => session("user").as[String].startsWith("admin")) { exec(goToAdminPage) } doIf doIfEquals doIfOrElse doSwitch doSwitchOrElse randomSwitch https://en.wikipedia.org/wiki/Decision_tree
  37. 37. Gatling DSL - error management exec(sendMoney) .tryMax(10){ exec(checkIfMoneyReceived) } tryMax exitBlockOnFail exitHereIfFailed Alice Bob Kafka
  38. 38. Gatling DSL - setup setUp(myScenario .inject( nothingFor(4 seconds), atOnceUsers(10), rampUsers(10) over (5 seconds)) .protocols(httpConf)) .maxDuration(10 minutes) constantUsersPerSec rampUsersPerSec splitUsers heavisideUsers
  39. 39. Gatling DSL - setup setUp(myScenario .inject(atOnceUsers(10)) .protocols(httpConf)) .assertions( global.responseTime.max.lt(50), global.failedRequests.percent.is(0) ) http://englishthroughlaxas.blogspot.com/2015/07/531-24-expression-of-assertion-emphasis.html
  40. 40. Gatling DSL - setup setUp(myScenario .inject(atOnceUsers(10)) .protocols(httpConf)) .throttle( reachRps(100) in (30 second), holdFor(1 minute), jumpToRps(50), holdFor(2 hours) )
  41. 41. Gatling DSL - feeders val companies = List("apple", "lenovo", "hp") val feeder = Iterator.continually( Map("company" -> companies(Random.nextInt(companies.size)))) val searching = scenario("Searching") .feed(feeder) .exec(searchFor("${company}")) RecordSeqFeederBuilder CSV JSON JDBC Sitemap Redis … http://favim.com/orig/201104/23/Favim.com-22725.jpg
  42. 42. Gatling DSL - resource inferring val httpConf = http .baseURL("http://computer-database.gatling.io") .acceptHeader("...") .acceptEncodingHeader("...") .acceptLanguageHeader("...") .inferHtmlResources() .userAgentHeader("...")
  43. 43. Gatling - launching ● gatling:test ● gatling:testOnly x.y.z.GetUserSimulation
  44. 44. Gatling DSL - other goodies Custom validators HTTP SSL SSE (Server Sent Event) basic cookies support WebSocket JMS Pluginable architecture: cassandra plugin kafka plugin rabbitMQ AMQP
  45. 45. Gatling DSL - logging/debugging ● logging ALL HTTP request and responses <logger name="io.gatling.http.ahc" level="TRACE" /> <logger name="io.gatling.http.response" level="TRACE" /> logging ONLY FAILED HTTP request and responses <logger name="io.gatling.http.ahc" level="DEBUG" /> <logger name="io.gatling.http.response" level="DEBUG" />
  46. 46. Gatling DSL - reporting ================================================================================ ---- Global Information -------------------------------------------------------- > request count 10 (OK=10 KO=0 ) > min response time 40 (OK=40 KO=- ) > max response time 177 (OK=177 KO=- ) > mean response time 55 (OK=55 KO=- ) > std deviation 41 (OK=41 KO=- ) > response time 50th percentile 42 (OK=42 KO=- ) > response time 75th percentile 43 (OK=43 KO=- ) > response time 95th percentile 117 (OK=117 KO=- ) > response time 99th percentile 165 (OK=165 KO=- ) > mean requests/sec 0.909 (OK=0.909 KO=- ) ---- Response Time Distribution ------------------------------------------------ > t < 800 ms 10 (100%) > 800 ms < t < 1200 ms 0 ( 0%) > t > 1200 ms 0 ( 0%) > failed 0 ( 0%) ================================================================================
  47. 47. Gatling DSL - distributed
  48. 48. Gatling DSL - distributed IT system
  49. 49. Gatling DSL - distributed ● manually ● Gatling FrontLine ● Flood.io IT system

×