121 Test Automation Day
Dublin, May 23rd 2018
Stating the obvious: adding performance and scalability tests to a Continuous Integration pipeline
Performance and scalability are core quality attributes of any system; unit testing, integration testing, UI testing, they all focus on functional requirements. Good performances mean happy users, less resource usage which translates to lower running costs (power, cloud bills) and customer retention.
In this session we will recall some basic concept of performance testing and demonstrate some of the many tools available in the cloud.
Performance is one of those areas that you could spend an infinite amount of time on, so failing to recognise it as a business interest could mean either chronic under-investment in it – or massive amounts of invisible over-spending if the technical team have placed too much emphasis on it. – Steve Fenton https://www.stevefenton.co.uk/2016/06/performance-is-a-feature/
Stating the obvious - 121 Test Automation Day, Dublin, 2018
Stating the Obvious
Adding Performance and Scalability Tests
to a Continuous Integration Pipeline
121 Test Automation Day
Dublin, 23 May 2018
Glass Lewis & Co.
Why is it worth?
What are the basics (aka theory)?
How to do it?
Bibliography at end
SlideShare after the
Bio in pictures
1KB RAM (upg. 16KB)
«there are two
kinds of websites:
the quick and the
0.1 second is about the limit for having the user
feel that the system is reacting instantaneously,
meaning that no special feedback is necessary
except to display the result.
1.0 second is about the limit for the user's flow of
thought to stay uninterrupted, even though the
user will notice the delay. Normally, no special
feedback is necessary during delays of more than
0.1 but less than 1.0 second, but the user does lose
the feeling of operating directly on the data.
10 seconds is about the limit for keeping the user's
attention focused on the dialogue. For longer
delays, users will want to perform other tasks
while waiting for the computer to finish, so they
should be given feedback indicating when the
computer expects to be done. Feedback during
the delay is especially important if the response
time is likely to be highly variable, since users will
then not know what to expect.
Miller, R. B. (1968).
Response time in man-
transactions. Proc. AFIPS
Fall Joint Computer
Conference Vol. 33, 267-
Half a second delay
caused a 20% drop in
Even very small delays
(100 ms) would result in
substantial and costly
drops in revenue.
Do you like your
Writing High-Performance .NET Code —
Ben Watson (Ben Watson)
Time Is Money: The Business Value of
Web Performance — Tammy Everts
Software Performance and Scalability: A
Quantitative Approach — Henry H. Liu
Continuous Delivery with Windows and .NET
— Matthew Skelton and Chris O'Dell