Learn how to write scripted load tests in PHP to run against your system without breaking the bank. Jason will cover not only the importance of load testing, but share stories of how load tests uncovered problems that would otherwise not have been discovered until production. Also, learn how to use load testing to learn how to deal with large traffic sites without needing to be employed by a large scale site first. We’ll be using RedLine13, an almost free load testing tool that is at the same time inexpensive, easy, and effective.

1. Load Testingwith Jason Lotito
“Guaranteed to be the best talk on load testing
with PHP @ MidwestPHP 2016 during that time
slot in that room with me as the speaker"

2. Jason Lotito
• DevOps Lead Architect
• MeetMe.com
• @jasonlotito
• jasonlotito.com
• jlotito@meetme.com

3. Lorem Ipsum Dolor
meetme.com/apps/careers

4. Load Testing
Not Unit Testing, Integration Testing, Acceptance
Testings, manual Testing, or Standardized Testing

5. Jason Lotito
Load test, or your users will do it for you.

6. Load test, or your users will do it for you.
Your Servers

7. Load test, or your users will do it for you.
And if they do, you won’t have any users.

8. You can use load testing to practice
dealing with lots of requests

9. What is Load Testing?

10. Load Testing
• Putting demand on a system
• Determine behavior under
load
• Determine maximum operating
capacity
• Discover Bottlenecks
• Determining steps to increase
operating capacity
• Optimize for costs

11. Other Load Testing Tools
• ApacheBench (ab)
• Apache JMeter
• siege
• BlazeMeter
• and many more…

12. The Basics of Load Testing

13. The Basics of Load Testing

14. Why RedLine13?
• Simple, up and running in little time
• Freemium, free level you only pay AWS
• Scriptable (PHP, Node.js)
• AWS for Infinite Scalability™
• I know Rich, one of the founders

15. RedLine13 has a bunch of features you can read
about on the website. I’m really here to talk to
you about the cool stuff I do with RedLine13.

20. Let’s walk through a
Custom Load Test

21. CustomTest.php
The class that actually runs the test.

22. In our example, we’ll use sleep to
simulate performing actions such as
making HTTP requests to an API

23. <?php
require_once( 'LoadTestingTest.class.php' );
class CustomTest extends LoadTestingTest
{
public function startTest()
{
for ( $x = 1; $x <= 100; $x++ )
{
$startTime = microtime(true);
sleep( mt_rand( 2, 5 ) );
$diff = microtime( true ) - $startTime;
recordPageTime( $startTime, $diff );
recordURLPageLoad( $x, $startTime, $diff );
}
return true;
}
}
CustomTest.php
This is
where your
code goes

24. <?php
require_once( 'LoadTestingTest.class.php' );
class CustomTest extends LoadTestingTest
{
public function startTest()
{
for ( $x = 1; $x <= 100; $x++ )
{
$startTime = microtime(true);
sleep( mt_rand( 2, 5 ) );
$diff = microtime( true ) - $startTime;
recordPageTime( $startTime, $diff );
recordURLPageLoad( $x, $startTime, $diff );
}
return true;
}
}
CustomTest.php

25. <?php
require_once( 'LoadTestingTest.class.php' );
class CustomTest extends LoadTestingTest
{
public function startTest()
{
for ( $x = 1; $x <= 100; $x++ )
{
$startTime = microtime(true);
sleep( mt_rand( 2, 5 ) );
$diff = microtime( true ) - $startTime;
recordPageTime( $startTime, $diff );
recordURLPageLoad( $x, $startTime, $diff );
}
return true;
}
}
CustomTest.php

26. <?php
require_once( 'LoadTestingTest.class.php' );
class CustomTest extends LoadTestingTest
{
public function startTest()
{
for ( $x = 1; $x <= 100; $x++ )
{
$startTime = microtime(true);
sleep( mt_rand( 2, 5 ) );
$diff = microtime( true ) - $startTime;
recordPageTime( $startTime, $diff );
recordURLPageLoad( $x, $startTime, $diff );
}
return true;
}
}
CustomTest.php

27. <?php
require_once( 'LoadTestingTest.class.php' );
class CustomTest extends LoadTestingTest
{
public function startTest()
{
for ( $x = 1; $x <= 100; $x++ )
{
$startTime = microtime(true);
sleep( mt_rand( 2, 5 ) );
$diff = microtime( true ) - $startTime;
recordPageTime( $startTime, $diff );
recordURLPageLoad( $x, $startTime, $diff );
}
return true;
}
}
CustomTest.php

28. <?php
require_once( 'LoadTestingTest.class.php' );
class CustomTest extends LoadTestingTest
{
public function startTest()
{
for ( $x = 1; $x <= 100; $x++ )
{
$startTime = microtime(true);
sleep( mt_rand( 2, 5 ) );
$diff = microtime( true ) - $startTime;
recordPageTime( $startTime, $diff );
recordURLPageLoad( $x, $startTime, $diff );
}
return true;
}
}
CustomTest.php

29. <?php
require_once( 'LoadTestingTest.class.php' );
class CustomTest extends LoadTestingTest
{
public function startTest()
{
for ( $x = 1; $x <= 100; $x++ )
{
$startTime = microtime(true);
sleep( mt_rand( 2, 5 ) );
$diff = microtime( true ) - $startTime;
recordPageTime( $startTime, $diff );
recordURLPageLoad( $x, $startTime, $diff );
}
return true;
}
}
CustomTest.php
Start Time
Start/End Time
Difference

30. <?php
require_once( 'LoadTestingTest.class.php' );
class CustomTest extends LoadTestingTest
{
public function startTest()
{
for ( $x = 1; $x <= 100; $x++ )
{
$startTime = microtime(true);
sleep( mt_rand( 2, 5 ) );
$diff = microtime( true ) - $startTime;
recordPageTime( $startTime, $diff );
recordURLPageLoad( $x, $startTime, $diff );
}
return true;
}
}
CustomTest.php
URL/String

31. These next two files
help us test locally

32. redline.php
Mock Redline functions to
test load test locally

33. <?php
/**
* @return bool
*/
function isRedlineVerbose()
{
return (int) getenv( 'REDLINE_VERBOSE' ) === 1;
}
/**
* @param int $timestamp
* @param int $elapsedTime
*/
function recordPageTime( $timestamp, $elapsedTime )
{
isRedlineVerbose() && output( "Record Page Time: timestamp
$timestamp; elapsed time: $elapsedTime" );
}
/**
redline.php

34. <?php
/**
* @return bool
*/
function isRedlineVerbose()
{
return (int) getenv( 'REDLINE_VERBOSE' ) === 1;
}
/**
* @param int $timestamp
* @param int $elapsedTime
*/
function recordPageTime( $timestamp, $elapsedTime )
{
isRedlineVerbose() && output( "Record Page Time: timestamp
$timestamp; elapsed time: $elapsedTime" );
}
/**
redline.php

35. <?php
/**
* @return bool
*/
function isRedlineVerbose()
{
return (int) getenv( 'REDLINE_VERBOSE' ) === 1;
}
/**
* @param int $timestamp
* @param int $elapsedTime
*/
function recordPageTime( $timestamp, $elapsedTime )
{
isRedlineVerbose() && output( "Record Page Time: timestamp
$timestamp; elapsed time: $elapsedTime" );
}
/**
* @param string $url
* @param int $timestamp
* @param int $elapsedTime
*/
function recordURLPageLoad( $url, $timestamp, $elapsedTime )
{
output( "$url -> $elapsedTime seconds" );
}
redline.php

36. * @param int $elapsedTime
*/
function recordPageTime( $timestamp, $elapsedTime )
{
isRedlineVerbose() && output( "Record Page Time: timestamp
$timestamp; elapsed time: $elapsedTime" );
}
/**
* @param string $url
* @param int $timestamp
* @param int $elapsedTime
*/
function recordURLPageLoad( $url, $timestamp, $elapsedTime )
{
output( "$url -> $elapsedTime seconds" );
}
/**
* @param int $kilobytes
*/
function recordDownloadSize( $kilobytes )
{
isRedlineVerbose() && output( sprintf( "Download Size: %dkb",
$kilobytes ) );
}
redline.php

37. * @param string $url
* @param int $timestamp
* @param int $elapsedTime
*/
function recordURLPageLoad( $url, $timestamp, $elapsedTime )
{
output( "$url -> $elapsedTime seconds" );
}
/**
* @param int $kilobytes
*/
function recordDownloadSize( $kilobytes )
{
isRedlineVerbose() && output( sprintf( "Download Size: %dkb",
$kilobytes ) );
}
/**
* @param string $error
*/
function recordError( $error )
{
output( "Error: $error" );
}
/**
* @param string $msg
redline.php

38. /**
* @param int $kilobytes
*/
function recordDownloadSize( $kilobytes )
{
isRedlineVerbose() && output( sprintf( "Download Size: %dkb",
$kilobytes ) );
}
/**
* @param string $error
*/
function recordError( $error )
{
output( "Error: $error" );
}
/**
* @param string $msg
* @return bool
*/
function output( $msg )
{
echo sprintf( "[%s] %s]n", getmypid(), $msg );
return true;
}
redline.php

39. }
/**
* @param string $error
*/
function recordError( $error )
{
output( "Error: $error" );
}
/**
* @param string $msg
* @return bool
*/
function output( $msg )
{
echo sprintf( "[%s] %s]n", getmypid(), $msg );
return true;
}
redline.php

40. run.php
The script we create to
test CustomTest.php locally

41. <?php
require_once('redline.php');
require_once('CustomTest.php');
$customTest = new CustomTest(1, mt_rand(1000000,9999999));
$customTest->startTest();
run.php

42. <?php
require_once('redline.php');
require_once('CustomTest.php');
$customTest = new CustomTest(1, mt_rand(1000000,9999999));
$customTest->startTest();
run.php

43. <?php
require_once('redline.php');
require_once('CustomTest.php');
$customTest = new CustomTest(1, mt_rand(1000000,9999999));
$customTest->startTest();
run.php

44. <?php
require_once('redline.php');
require_once('CustomTest.php');
$customTest = new CustomTest(1, mt_rand(1000000,9999999));
$customTest->startTest();
run.php
Instance
Number

45. <?php
require_once('redline.php');
require_once('CustomTest.php');
$customTest = new CustomTest(1, mt_rand(1000000,9999999));
$customTest->startTest();
run.php
User
Number
Random
Number

46. What does a result look like?

55. You can use error reporting to
simulate real time logging
Errors show up in real time, logs show up after
the test is completed. Use error reports to let you
know if you should stop your load test.

56. Sample App: Chat Service

57. Sample App: Chat Service
• 1-on-1 Chat
• Group Chat
• Browse Chat Rooms
• Search for Chat Room Topics

58. Why split 1-on-1 and group
chat into separate tests?

59. Multiple tests can be run at
the same time.

60. 1-on-1
Group
30-40%
60-70%
30-40%
60-70% 60-70%
30-40%
Time
Start at the
same time
Increase
group chat %
Could mean just more group chat or
both more group and less 1-on-1
Revert to
normal ratio
20:00 40:000:00

61. 1-on-1
Group
30-40%
60-70%
30-40%
60-70% 60-70%
30-40%
Time
Could be done in your
script or in RedLine13

62. How?
• Start 4 tests:
• For both group and 1-on-1:
• 350 users
• 300 users

63. 1-on-1
Group
350
350+300
350
350+300 350
350
Time
300 test sleeps
for 20 minutes
After 20 minutes,
it stops.

64. Or, instead of timers,
just do it manually.
Usually the easiest on the day of.

65. The point is, you can adjust
your load test as needed.

66. Performing Load Tests

67. Things You Need
• Production-level systems to test against
• Monitoring
• Automated systems to deploy changes
• Dedicated people
• A time to test (load testing day!)

68. Things You Need
• Goals, such as
• 500 requests per second
• 90th percentile @ 50ms

69. Planning the Load Test
• Start at what you expect normal traffic to be
• 500rps
• Start a second test that is just the burst traffic
• 900rps
• Ramps up over 20 mins
• Lasts 40 mins at peak
• Ramps down over 20 mins

70. Learning
• Is proper monitoring setup?
• Is a scaling plan in place? Can you scale?
• What happens to the app under load?
• What slows down first?
• Are things alerting as expected?

71. As Load Tester, you are responsible for…
• Ensuring those involved with the system under test are
aware and available
• Putting the system under test under load
• Completely destroying the system under load
• Providing a clear and concise write up of exactly what
occurred
• Take charge! It’s fun!

72. Reporting

73. Reporting Results: Shortest Version
• 99% @ 59ms

74. Reporting Results: Short Version
• Short version: During an hour starting with 0rps to 1400 rps
in the first 10 minutes....
• ...when starting with 5 instances and scaling to 11
instances, the response times were: 50% 23ms, 75% 54ms,
95% 303ms, and 99% 1069ms.
• ...when starting with 11 instances, the response times
were: 50% 16ms, 75% 24ms, 95% 45ms, and 99% 59ms.

75. Reporting Results: Detailed Version
• Provide more information
• Results
• Story driven
• Have data supporting results prepared in case
• Account for changes such as auto-scaling
• With auto-scaling: 99% at 1069ms
• Without auto-scaling: 99% at 59ms

76. Testing elasticsearch on c3.4xlarge
Detailed Reporting
Also included in this report
was a link to the actual
GitHub repository. Make sure
you are keeping your load
tests in version control!

77. Tag your load test in git
Associate the tag with a Load Test,
so you can replay any load test easily

78. Lessons Learned

79. Things to Keep In Mind
• Understand expected usage
• X% of users using the app while
• Y% are chatting with one another
• Users are logging in
• Creating accounts
• Backend systems
• Determine what’s important

80. Things to Keep In Mind
• User input
• Random filters
• Weighted filters
• Cached results are expected
• Client constraints

81. Things to Keep In Mind
• User flow through service
• Try to understand how users use the app
• Script should try to mimic

82. Things to Keep In Mind
• Be careful about testing a single system
• System will have logging
• System will have backend services
• You’d be surprised what can cause failure
• A load test helps you learn before it’s in production

83. Things to Keep In Mind
• User interaction
• MeetMe is social, so we’ve load tested chatting
• 1 test per 2 users, both chatting with one another

84. Things to Keep In Mind
• Have developers available
• Better, have developers with you when load testing

85. Things to Keep In Mind
• Find a problem, Fix it, Reroll app, Rerun test
• FFRR, or F2R2
• I just made that up
• Don’t use it.

86. Things to Keep In Mind
• Start testing from your laptop
• Seriously, my MacBook Air could bring down Erlang
services
• Database indexes are a thing
• While running a load test, you can run a single client form
your laptop

87. Things to Keep In Mind
• Someone should be testing the app/service as well
• Response times are only a number
• What does 50ms vs 300ms response times feel to the user
• What impact does 2x/3x/4x load have
• When auto-scaling, how does the client handle

88. Things to Keep In Mind
• Review how the client is using the API
• Review how the API developer expects the client to use the
API
• Model after what the client is doing
• Call out differences early

89. Learn
Most importantly…

90. Thanks!
• @jasonlotito
• jasonlotito.com
• jlotito@meetme.com
• https://joind.in/talk/80910