Performance tests with Gatling are difficult for three main reasons: 1) The test environment must closely simulate production in terms of hardware, software, and load. 2) Proper infrastructure for monitoring, logging, and isolating tests is required. 3) Performance intuition can be wrong, so statistics like percentiles must be used rather than averages.

1. Performance tests with
Gatling
Andrzej Ludwikowski

2. About me
➔aludwikowski.blogspot.com
➔github.com/aludwiko
➔@aludwikowski

4. Performance tests - why so hard?
Simulate production as close as possible:
hardware
CPU, RAM, storage, ...
software
OS, Virtualization, DBs, …
load
isolation

5. Performance tests - why so hard?
Necessary to have:
infrastructure
monitoring
logging

6. Performance tests - why so hard?
Your performance intuition is wrong!

7. Performance tests - why so hard?
Lies, damned lies, and statistics:
arithmetic mean = 2.9
median = 1
standard deviation = 6 (only for normal distribution)
Use:
percentiles
50 = 1
70 = 1
90 = 2.9
95 = 11.45
99 = 18.29
Check percentiles implementation!

8. Performance tests - why so hard?
● Coordinated omission problem by Gil Tene
http://bravenewgeek.com/tag/coordinated-omission/

9. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)

10. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)
samples 31
50 % 1
70 % 1
90 % 1
95 % 1
99 % 12,89
99,9 % 28,30
99,99 % 29,82
avg 1,93

11. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)
samples 10001
50 % 1
70 % 1
90 % 1
95 % 1
99 % 1
99,9 % 1,15
99,99 % 28,01
avg 1,03

12. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)

13. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/
system.exit(0)
samples 60
50 % 1
70 % 12,6
90 % 24,3
95 % 27,2
99 % 29,45
99,9 % 29,95
99,99 % 29,99
avg 8,25

14. Performance tests - why so hard?

15. Performance tests - why so hard?
total time 60 s
max 30 s
99% 1 s

16. Performance tests - why so hard?
total time 60 s
max 30 s
99% 1 s
time in % for max 50%

17. Performance tests - why so hard?
total time 60 s
max 30 s
99% 1 s
time in % for max 50%
expected time in % for 99% 50% - 1% = 49%

18. Performance tests - why so hard?
total time 60 s
max 30 s
99% 1 s
time in % for max 50%
expected time in % for 99% 50% - 1% = 49%
real time for 99% 60 s * 49% = 29,4 s

19. Performance tests - why so hard?
http://bravenewgeek.com/tag/coordinated-omission/

20. Performance tests - why so hard?
Tests…

21. Performance tests - why so hard?
Tests…

22. Gatling

23. Gatling

24. Why Gatling?
non-blocking, asynchronous stack (scala, akka, netty)
scala !!!111oneoneone (maven, sbt support)
DSL
recorder
math is good
reports
integration & performance tests

25. Gatling DSL
class BasicSimulation extends Simulation {
val httpConf = http
.baseURL("http://computer-database.gatling.io")
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
.acceptEncodingHeader("gzip, deflate")
.acceptLanguageHeader("en-US,en;q=0.5")
.userAgentHeader("Mozilla/5.0 (Macintosh; X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0")
val firstScenario = scenario("First Scenario Name")
.exec(http("Request name").get("/"))
.pause(7)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

26. Gatling DSL
class FirstScenarioV2 extends MySimulation {
val firstScenario = scenario("First Scenario Name")
.exec(http("Go to root page").get("/"))
.pause(7 seconds)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

27. Gatling DSL
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class FirstScenarioV2 extends MySimulation {
val firstScenario = scenario("First Scenario Name")
.exec(http("Go to root page").get("/"))
.pause(7 seconds)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

28. Gatling DSL
class ComplexScenario extends MySimulation {
val complexScenario = scenario("Complex demo scenario")
.exec(http("request_1").get("/")).pause(7)
.exec(http("request_2").get("/computers?f=macbook")).pause(2)
.exec(http("request_3").get("/computers/6")).pause(3)
.exec(http("request_4").get("/")).pause(2)
.exec(http("request_5").get("/computers?p=1")).pause(670 milliseconds)
.exec(http("request_6").get("/computers/new")).pause(1)
.exec(http("request_7")
.post("/computers")
.formParam("name", "MyComputer").formParam("introduced", "2012-05-30").formParam("company", "37"))
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

29. Gatling DSL
class ComplexScenarioV2 extends MySimulation {
val complexScenario = scenario("Complex demo scenario")
.exec(goToRootPage).pause(7)
.exec(searchFor("macbook")).pause(2)
.exec(positionAt(6)).pause(3)
.exec(goToRootPage).pause(2)
.exec(goToPage(1)).pause(670 milliseconds)
.exec(openNewComputerForm).pause(1)
.exec(addNewComputer)
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

30. Gatling DSL
class ComplexScenarioV3 extends MySimulation {
val search = exec(goToRootPage).pause(7)
.exec(searchFor("macbook")).pause(2)
.exec(positionAt(6)).pause(3)
val addComputer = exec(goToRootPage).pause(2)
.exec(goToPage(1)).pause(670 milliseconds)
.exec(openNewComputerForm).pause(1)
.exec(addNewComputer)
val complexScenario = scenario("Complex demo scenario").exec(search, addComputer)
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}

31. Gatling DSL - checks
scenario("DSL demo")
.exec(http("go to page")
.get("/computers")
.check(status.is(200))
.check(status.in(200 to 210))

32. Gatling DSL - checks
scenario("DSL demo")
.exec(http("go to page")
.get("/computers")
.check(regex("computers")
.find(1)
.exists)
https://www.pinterest.com/pin/491666484294006138/

33. Gatling DSL - checks
scenario("First Scenario Name")
.exec(http("Request name")
.get("/computers")
.check(jsonPath("$..foo.bar[2].baz").ofType[String].notExists)
.check(xpath("//input[@id='text1']/@value").is("test"))
.check(css("...").transform(_.split('|').toSeq).is(Seq("1", "2")))

34. Gatling DSL - checks
scenario("DSL demo")
.exec(http("Authorize")
.get("/auth")
.check(regex("token").find(1).exists
.saveAs("authorizationToken")))
.exec(http("Authorized resource")
.get("/authorized_resource?token=${authorizationToken}"))

35. Gatling DSL - looping
repeat(5, "i") {
exec(goToPage("${i}".toInt))
.pause(1)
}
repeat
foreach
during
asLongAs
forever
https://blog.hubspot.com/blog/tabid/6307/bid/32019/Why-Every-Marketer-Needs-Closed-Loop-Reporting.aspx#sm.0005lrqj811waf3ntmn1cul3881gr

36. Gatling DSL - polling
exec(
polling
.every(10 seconds)
.exec(searchFor("thinkpad"))
)
http://www.firmus-solutions.com/terms-conditions.html

37. Gatling DSL - conditions
doIf(session => session("user").as[String].startsWith("admin")) {
exec(goToAdminPage)
}
doIf
doIfEquals
doIfOrElse
doSwitch
doSwitchOrElse
randomSwitch
https://en.wikipedia.org/wiki/Decision_tree

38. Gatling DSL - error management
exec(sendMoney)
.tryMax(10){
exec(checkIfMoneyReceived)
}
tryMax
exitBlockOnFail
exitHereIfFailed
Alice Bob
Kafka

39. Gatling DSL - setup
setUp(myScenario
.inject(
nothingFor(4 seconds),
atOnceUsers(10),
rampUsers(10) over (5 seconds))
.protocols(httpConf))
.maxDuration(10 minutes)
constantUsersPerSec
rampUsersPerSec
splitUsers
heavisideUsers

40. Gatling DSL - setup
setUp(myScenario
.inject(atOnceUsers(10))
.protocols(httpConf))
.assertions(
global.responseTime.max.lt(50),
global.failedRequests.percent.is(0)
)
http://englishthroughlaxas.blogspot.com/2015/07/531-24-expression-of-assertion-emphasis.html

41. Gatling DSL - setup
setUp(myScenario
.inject(atOnceUsers(10))
.protocols(httpConf))
.throttle(
reachRps(100) in (30 second),
holdFor(1 minute),
jumpToRps(50),
holdFor(2 hours)
)

42. Gatling DSL - feeders
val companies = List("apple", "lenovo", "hp")
val feeder = Iterator.continually(
Map("company" -> companies(Random.nextInt(companies.size))))
val searching = scenario("Searching")
.feed(feeder)
.exec(searchFor("${company}"))
RecordSeqFeederBuilder
CSV
JSON
JDBC
Sitemap
Redis
… http://favim.com/orig/201104/23/Favim.com-22725.jpg

43. Gatling DSL - resource inferring
val httpConf = http
.baseURL("http://computer-database.gatling.io")
.acceptHeader("...")
.acceptEncodingHeader("...")
.acceptLanguageHeader("...")
.inferHtmlResources()
.userAgentHeader("...")

44. Gatling - launching
● gatling:test
● gatling:testOnly x.y.z.GetUserSimulation

45. Gatling DSL - other goodies
Custom validators
HTTP
SSL
SSE (Server Sent Event)
basic cookies support
WebSocket
JMS
Pluginable architecture:
cassandra plugin
kafka plugin
rabbitMQ
AMQP

46. Gatling DSL - logging/debugging
● logging ALL HTTP request and responses
<logger name="io.gatling.http.ahc" level="TRACE" />
<logger name="io.gatling.http.response" level="TRACE" />
logging ONLY FAILED HTTP request and responses
<logger name="io.gatling.http.ahc" level="DEBUG" />
<logger name="io.gatling.http.response" level="DEBUG" />

47. Gatling DSL - reporting
================================================================================
---- Global Information --------------------------------------------------------
> request count 10 (OK=10 KO=0 )
> min response time 40 (OK=40 KO=- )
> max response time 177 (OK=177 KO=- )
> mean response time 55 (OK=55 KO=- )
> std deviation 41 (OK=41 KO=- )
> response time 50th percentile 42 (OK=42 KO=- )
> response time 75th percentile 43 (OK=43 KO=- )
> response time 95th percentile 117 (OK=117 KO=- )
> response time 99th percentile 165 (OK=165 KO=- )
> mean requests/sec 0.909 (OK=0.909 KO=- )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms 10 (100%)
> 800 ms < t < 1200 ms 0 ( 0%)
> t > 1200 ms 0 ( 0%)
> failed 0 ( 0%)
================================================================================

48. Gatling DSL - distributed

49. Gatling DSL - distributed
IT system

50. Gatling DSL - distributed
● manually
● Gatling FrontLine
● Flood.io
IT system