More than just giving Apache benchmark a url to request many times, RedLine13 lets you script yourself actual users, mimicking real requests to your system. This not only lets you discover the where your bottlenecks are, but lets you to actually test different scenarios while under real load. Jason will cover not only why you would want to load test a system, but will also cover what you want to look for when load testing, as well as show you how to write load tests to use with RedLine13. RedLine13 is an inexpensive load testing tool that turns load testing from a chore into something you'll recommend at every point. After all, who doesn't like watching things crash and burn, and getting paid to do it?

1. Or getting paid to DoS your own systems
Load Testing with
RedLine13
Jason Lotito

2. What is Load Testing?

3. Load testing is the process of putting demand on
a software system or computing device and
measuring its response.

4. Load testing is performed to determine a
system's behavior under both normal and
anticipated peak load conditions.

5. It helps to identify the maximum operating
capacity of an application as well as any
bottlenecks and determine which element is
causing degradation.

6. What about Stress Testing?

7. Stress testing focuses on throughput.
Load testing focuses on response times.
Both can be performed at the same time.

8. Load Testing
• Putting demand on a system
• Measuring the response
• Determine behavior under
load
• Determine maximum operating
capacity
• Bottlenecks

9. Other Load Testing Tools
• ApacheBench (ab)
• Apache JMeter
• siege
• BlazeMeter

10. Why RedLine13?
• Simple, up and running in little time
• Inexpensive, you pay for what you use
• Scriptable, so you have complete control over your
tests
• I know Rich

11. RedLine13 has a bunch of features you can read
about on the website. I’m really here to talk to
you about the cool stuff I do with RedLine13.

12. The Basics of Load Testing
The Basics of RedLine13
This is another subtitle that I’m
apparently supposed to use, in place of
the title, and the super subtitle above
the title.

19. Preparing for Load Testing

20. Need
• Production-level systems to test against
• Monitoring
• CPU, memory, I/O, etc.
• Application dashboards
• Logging
• Automated systems to deploy
• Dedicated people
• A time to test

21. Learning
• Set goals, e.g.
• 50ms
• 500rps
• Test monitoring
• Scaling
• What happens to the app under load?
• What slows down first?

22. As Load Tester, you are responsible for…
• Ensuring those involved with the system under test are
aware and available
• Putting the system under test under load
• Completely destroying the system under load
• Providing a clear and concise write up of exactly what
occurred
• Take charge! It’s fun!

23. Ramp Up Time
• Start at what you expect normal traffic to be
• 500rps
• Start a second test that is just the burst traffic
• 900rps
• Ramps up over 20 mins
• Lasts 40 mins at peak
• Ramps down over 20 mins

24. Reporting

25. Reporting Results: Shortest Version
• 99% @ 59ms

26. Reporting Results: Short Version
• Short version: During an hour starting with 0rps to 1400 rps
in the first 10 minutes....
• ...when starting with 5 instances and scaling to 11
instances, the response time were: 50% 23ms, 75% 54ms,
95% 303ms, and 99% 1069ms.
• ...when starting with 11 instances, the response times
were: 50% 16ms, 75% 24ms, 95% 45ms, and 99% 59ms.

27. Reporting Results: Detailed Version
• Provide more information
• Results
• Story driven
• Have data supporting results prepared in case
• Account for changes such as auto-scaling
• With auto-scaling: 99% at 1069ms
• Without auto-scaling: 99% at 59ms

28. Testing elasticsearch on c3.4xlarge
Detailed Reporting
Also included in this report was a link to
the actual GitHub repository. Make sure
you are keeping your load tests in
version control!

31. Tag your load test in git
Associate the tag with a Load Test,
so you can replay any load test easily

32. PHP & Node.js
We’ll be using Node.js for our examples

33. Writing a RedLine13 Custom Load Test in Node.js
• Uploading a *.tar.gz/*.zip file
• npm install is run
• Expects a CustomTest.js file in root directory
• CustomTest.js should export a ‘class’
• Will create a new object from
• class(redLine, testNum, rand, config)
• Will call runTest() on object of class

34. {
"name": "reloaded",
"version": "1.0.0",
"description": "Recommendation Engine Load Tester",
"main": "app.js",
"scripts": {
"test": "echo "Error: no test specified" && exit 1",
"start": "node app.js",
"build_test": "rm reloaded.tar.gz && tar --exclude='node_modules' --
exclude='.git' --exclude='logs' -cvzf reloaded.tar.gz *"
},
"author": "Jason Lotito <jlotito@meetme.com>",
"license": "ISC",
"dependencies": {
"async": "^1.3.0",
"elasticsearch": "^5.0.0",
"lodash": "^3.10.0",
package.json

35. {
"name": "reloaded",
"version": "1.0.0",
"description": "Recommendation Engine Load Tester",
"main": "app.js",
"scripts": {
"test": "echo "Error: no test specified" && exit 1",
"start": "node app.js",
"build_test": "rm reloaded.tar.gz && tar --exclude='node_modules' --
exclude='.git' --exclude='logs' -cvzf reloaded.tar.gz *"
},
"author": "Jason Lotito <jlotito@meetme.com>",
"license": "ISC",
"dependencies": {
"async": "^1.3.0",
"elasticsearch": "^5.0.0",
"lodash": "^3.10.0",
package.json

36. {
"name": "reloaded",
"version": "1.0.0",
"description": "Recommendation Engine Load Tester",
"main": "app.js",
"scripts": {
"test": "echo "Error: no test specified" && exit 1",
"start": "node app.js",
"build_test": "rm reloaded.tar.gz && tar --exclude='node_modules' --
exclude='.git' --exclude='logs' -cvzf reloaded.tar.gz *"
},
"author": "Jason Lotito <jlotito@meetme.com>",
"license": "ISC",
"dependencies": {
"async": "^1.3.0",
"elasticsearch": "^5.0.0",
"lodash": "^3.10.0",
package.json

37. var Test = require('./CustomTest');
var rl = require('./lib/RedLine');
(new Test(new rl.RedLine, 55)).runTest(function(){
console.log( "test complete");
});
app.js

38. var Test = require('./CustomTest');
var rl = require('./lib/RedLine');
(new Test(new rl.RedLine, 55)).runTest(function(){
console.log( "test complete");
});
app.js

39. var Test = require('./CustomTest');
var rl = require('./lib/RedLine');
(new Test(new rl.RedLine, 55)).runTest(function(){
console.log( "test complete");
});
app.js

40. var Test = require('./CustomTest');
var rl = require('./lib/RedLine');
(new Test(new rl.RedLine, 55)).runTest(function(){
console.log( "test complete");
});
app.js

41. function RedLine(){}
RedLine.prototype.recordPageTime = function(ts, time){};
RedLine.prototype.recordURLPageLoad = function(url, ts, time){};
RedLine.prototype.recordDownloadSize = function(kb){};
RedLine.prototype.recordError = function(error){console.log(error)};
RedLine.prototype.recordProgress = function(testNum, percent){};
function Timer( url, redLineAPI )
{
this.url = url;
this.redLine = redLineAPI;
}
RedLine.js

42. function RedLine(){}
RedLine.prototype.recordPageTime = function(ts, time){};
RedLine.prototype.recordURLPageLoad = function(url, ts, time){};
RedLine.prototype.recordDownloadSize = function(kb){};
RedLine.prototype.recordError = function(error){console.log(error)};
RedLine.prototype.recordProgress = function(testNum, percent){};
function Timer( url, redLineAPI )
{
this.url = url;
this.redLine = redLineAPI;
}
RedLine.js

43. function RedLine(){}
RedLine.prototype.recordPageTime = function(ts, time){};
RedLine.prototype.recordURLPageLoad = function(url, ts, time){};
RedLine.prototype.recordDownloadSize = function(kb){};
RedLine.prototype.recordError = function(error){console.log(error)};
RedLine.prototype.recordProgress = function(testNum, percent){};
function Timer( url, redLineAPI )
{
this.url = url;
this.redLine = redLineAPI;
}
RedLine.js

44. function RedLine(){}
RedLine.prototype.recordPageTime = function(ts, time){};
RedLine.prototype.recordURLPageLoad = function(url, ts, time){};
RedLine.prototype.recordDownloadSize = function(kb){};
RedLine.prototype.recordError = function(error){console.log(error)};
RedLine.prototype.recordProgress = function(testNum, percent){};
function Timer( url, redLineAPI )
{
this.url = url;
this.redLine = redLineAPI;
}
RedLine.js

45. function RedLine(){}
RedLine.prototype.recordPageTime = function(ts, time){};
RedLine.prototype.recordURLPageLoad = function(url, ts, time){};
RedLine.prototype.recordDownloadSize = function(kb){};
RedLine.prototype.recordError = function(error){console.log(error)};
RedLine.prototype.recordProgress = function(testNum, percent){};
function Timer( url, redLineAPI )
{
this.url = url;
this.redLine = redLineAPI;
}
RedLine.js

46. function RedLine(){}
RedLine.prototype.recordPageTime = function(ts, time){};
RedLine.prototype.recordURLPageLoad = function(url, ts, time){};
RedLine.prototype.recordDownloadSize = function(kb){};
RedLine.prototype.recordError = function(error){console.log(error)};
RedLine.prototype.recordProgress = function(testNum, percent){};
function Timer( url, redLineAPI )
{
this.url = url;
this.redLine = redLineAPI;
}
RedLine.js

47. RedLine.prototype.recordError = function(error){console.log(error)};
RedLine.prototype.recordProgress = function(testNum, percent){};
function Timer( url, redLineAPI )
{
this.url = url;
this.redLine = redLineAPI;
}
Timer.prototype.start = function ()
{
return new ActiveTimer( Date.now(), this.url, this.redLine );
};
function ActiveTimer( startTime, url, redLineAPI )
{
RedLine.js

48. RedLine.prototype.recordError = function(error){console.log(error)};
RedLine.prototype.recordProgress = function(testNum, percent){};
function Timer( url, redLineAPI )
{
this.url = url;
this.redLine = redLineAPI;
}
Timer.prototype.start = function ()
{
return new ActiveTimer( Date.now(), this.url, this.redLine );
};
function ActiveTimer( startTime, url, redLineAPI )
{
RedLine.js

49. Timer.prototype.start = function ()
{
return new ActiveTimer( Date.now(), this.url, this.redLine );
};
function ActiveTimer( startTime, url, redLineAPI )
{
this.endTime = 0;
this.startTime = startTime;
this.redLine = redLineAPI;
this.url = url;
}
ActiveTimer.prototype.stop = function ()
{
this.endTime = Date.now();
RedLine.js

50. this.redLine = redLineAPI;
this.url = url;
}
ActiveTimer.prototype.stop = function ()
{
this.endTime = Date.now();
var timeDiff = this.endTime - this.startTime;
this.redLine.recordError(timeDiff);
this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );
this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );
return timeDiff;
};
module.exports= {
RedLine: RedLine,
RedLine.js

51. this.redLine = redLineAPI;
this.url = url;
}
ActiveTimer.prototype.stop = function ()
{
this.endTime = Date.now();
var timeDiff = this.endTime - this.startTime;
this.redLine.recordError(timeDiff);
this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );
this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );
return timeDiff;
};
module.exports= {
RedLine: RedLine,
RedLine.js

52. this.redLine = redLineAPI;
this.url = url;
}
ActiveTimer.prototype.stop = function ()
{
this.endTime = Date.now();
var timeDiff = this.endTime - this.startTime;
this.redLine.recordError(timeDiff);
this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );
this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );
return timeDiff;
};
module.exports= {
RedLine: RedLine,
RedLine.js

53. this.redLine = redLineAPI;
this.url = url;
}
ActiveTimer.prototype.stop = function ()
{
this.endTime = Date.now();
var timeDiff = this.endTime - this.startTime;
this.redLine.recordError(timeDiff);
this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );
this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );
return timeDiff;
};
module.exports= {
RedLine: RedLine,
RedLine.js

54. this.redLine = redLineAPI;
this.url = url;
}
ActiveTimer.prototype.stop = function ()
{
this.endTime = Date.now();
var timeDiff = this.endTime - this.startTime;
this.redLine.recordError(timeDiff);
this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );
this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );
return timeDiff;
};
module.exports= {
RedLine: RedLine,
RedLine.js

55. var timeDiff = this.endTime - this.startTime;
this.redLine.recordError(timeDiff);
this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );
this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );
return timeDiff;
};
module.exports= {
RedLine: RedLine,
Timer: Timer
};
RedLine.js

56. var async = require( 'async' );
var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
CustomTest.js

57. var async = require( 'async' );
var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
CustomTest.js

58. var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
function rand( low, high )
{
return Math.floor( Math.random() * (high - low + 1) + low );
}
CustomTest.js

59. var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
function rand( low, high )
{
return Math.floor( Math.random() * (high - low + 1) + low );
}
CustomTest.js

60. var sleep = require( 'sleep' );
var Timer = require( './lib/RedLine' ).Timer;
var Test = require( './lib/Test' );
// Load Testing Options
var MINUTES_TO_RUN = 60;
var MAX_MINUTES_TO_WAIT_BEFORE_START = 10;
var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
function rand( low, high )
{
return Math.floor( Math.random() * (high - low + 1) + low );
}
CustomTest.js

61. var MIN_MINUTES_TO_WAIT_BEFORE_START = 0;
// Normally don't need to change
var RUN_EVERY_MILLISECONDS = 1000;
function rand( low, high )
{
return Math.floor( Math.random() * (high - low + 1) + low );
}
function Reloaded( rl, testNum, rand, config )
{
this.rl = rl;
this.testNum = testNum;
this.rand = rand;
this.config = config;
CustomTest.js

62. {
return Math.floor( Math.random() * (high - low + 1) + low );
}
function Reloaded( rl, testNum, rand, config )
{
this.rl = rl;
this.testNum = testNum;
this.rand = rand;
this.config = config;
this.test = new Test();
this.rampUp();
}
Reloaded.prototype.rampUp = function ()
CustomTest.js

63. {
return Math.floor( Math.random() * (high - low + 1) + low );
}
function Reloaded( rl, testNum, rand, config )
{
this.rl = rl;
this.testNum = testNum;
this.rand = rand;
this.config = config;
this.test = new Test();
this.rampUp();
}
Reloaded.prototype.rampUp = function ()
CustomTest.js

64. {
return Math.floor( Math.random() * (high - low + 1) + low );
}
function Reloaded( rl, testNum, rand, config )
{
this.rl = rl;
this.testNum = testNum;
this.rand = rand;
this.config = config;
this.test = new Test();
this.rampUp();
}
Reloaded.prototype.rampUp = function ()
CustomTest.js

65. }
function Reloaded( rl, testNum, rand, config )
{
this.rl = rl;
this.testNum = testNum;
this.rand = rand;
this.config = config;
this.test = new Test();
this.rampUp();
}
Reloaded.prototype.rampUp = function ()
{
var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60,
MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );
CustomTest.js

66. this.test = new Test();
this.rampUp();
}
Reloaded.prototype.rampUp = function ()
{
var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60,
MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );
console.log( Date.now() + " - Sleeping for " + sleepInSeconds );
sleep.sleep( sleepInSeconds );
console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
CustomTest.js

67. this.test = new Test();
this.rampUp();
}
Reloaded.prototype.rampUp = function ()
{
var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60,
MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );
console.log( Date.now() + " - Sleeping for " + sleepInSeconds );
sleep.sleep( sleepInSeconds );
console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
CustomTest.js

68. this.test = new Test();
this.rampUp();
}
Reloaded.prototype.rampUp = function ()
{
var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60,
MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );
console.log( Date.now() + " - Sleeping for " + sleepInSeconds );
sleep.sleep( sleepInSeconds );
console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
CustomTest.js

69. function Reloaded( rl, testNum, rand, config )
{
this.rl = rl;
this.testNum = testNum;
this.rand = rand;
this.config = config;
this.test = new Test();
this.rampUp();
}
Reloaded.prototype.rampUp = function ()
{
var sleepInSeconds = rand( MIN_MINUTES_TO_WAIT_BEFORE_START * 60,
MAX_MINUTES_TO_WAIT_BEFORE_START * 60 );
console.log( Date.now() + " - Sleeping for " + sleepInSeconds );
CustomTest.js

70. console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
var max = 60 * MINUTES_TO_RUN;
async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
CustomTest.js

71. console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
var max = 60 * MINUTES_TO_RUN;
async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
CustomTest.js

72. console.log( Date.now() + " - Done sleeping for " + sleepInSeconds );
};
Reloaded.prototype.runTest = function ( finished )
{
var timer = new Timer( 'test', this.rl );
var max = 60 * MINUTES_TO_RUN;
async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
CustomTest.js

73. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js

74. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js

75. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js

76. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js

77. async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
CustomTest.js

78. var max = 60 * MINUTES_TO_RUN;
async.timesSeries( max, function ( n, next )
{
var activeTimer = new timer.start();
var timeDiff = 0;
this.test.runATest( function ( err )
{
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
CustomTest.js

79. {
if ( err )
{
this.rl.recordError( err.message );
}
timeDiff = activeTimer.stop();
setTimeout( next, RUN_EVERY_MILLISECONDS - timeDiff );
}.bind( this ) );
}.bind( this ), function ()
{
finished( true );
} );
};
CustomTest.js

80. Now, let’s hack logging
By default,RedLine doesn’t support logging
through the web UI

81. this.redLine = redLineAPI;
this.url = url;
}
ActiveTimer.prototype.stop = function ()
{
this.endTime = Date.now();
var timeDiff = this.endTime - this.startTime;
this.redLine.recordError(timeDiff);
this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );
this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );
return timeDiff;
};
module.exports= {
RedLine: RedLine,
RedLine.js

82. this.redLine = redLineAPI;
this.url = url;
}
ActiveTimer.prototype.stop = function ()
{
this.endTime = Date.now();
var timeDiff = this.endTime - this.startTime;
this.redLine.recordError(timeDiff);
this.redLine.recordPageTime( this.startTime/1000, timeDiff/1000 );
this.redLine.recordURLPageLoad( this.url, this.startTime/1000, timeDiff/1000 );
return timeDiff;
};
module.exports= {
RedLine: RedLine,
RedLine.js

84. Response Times
Counts

86. 19 requests at 27ms

87. With this we can get percentiles
Of course, the pro version offers this for PHP

91. You can use error reporting to
simulate real time logging
The important thing here is…

92. Writing Custom Load Tests

93. Things to Keep In Mind
• Understand expected usage
• X% of users using the app while
• Y% are chatting with one another
• Users are logging in
• Creating accounts
• Backend systems
• Determine what’s important

94. Things to Keep In Mind
• User input
• Random filters
• Weighted filters
• Cached results are expected
• Client constraints

95. Things to Keep In Mind
• User flow through service
• Try to understand how users use the app
• Script should try to mimic

96. Things to Keep In Mind
• Be care about testing a single system
• System will have logging
• System will have backend services
• You’d be surprised what can cause failure
• A Load Test helps you learn before it’s in production

97. Things to Keep In Mind
• User interaction
• MeetMe is social, so we’ve load tested chatting
• 1 test per 2 users, both chatting with one another
• Story time!

98. Things to Keep In Mind
• Have developers available
• Better, have developers with you when load testing
• Find a problem, Fix it, Reroll app, Rerun test
• FFRR
• I just made that up
• Don’t use it.

99. Things to Keep In Mind
• Start testing from your laptop
• Seriously, my MacBook Air could bring down Erlang
services
• Database indexes are a thing
• While running a load test, you can run a single client
form your laptop

100. Things to Keep In Mind
• Someone should be testing the app/service as well
• Response times are only a number
• What does 50ms vs 300ms response times feel to the
user
• What impact does 2x/3x/4x load have
• When auto-scaling, how does the client handle

101. Things to Keep In Mind
• Review how the client is using the API
• Review how the API developer expects the client to use
the API
• Model after what the client is doing
• Call out differences early

102. Learn
Most importantly…

103. In Russia, Load tests You!
Good load tests will teach you things
you didn’t know about your systems.
Great load tests will help you solve those problems
before they become production problems.

104. Load Testing with RedLine13
Thanks!
@jasonlotito
jasonlotito.com
jasonlotito@gmail.com