The Value of Testing in the Web World
Does your organization measure the value of web application testing? Or does
testing become the hot topic of discussion only when there is an application failure or
a website visitor files a complaint to customer service? Is that when QA is called in to
quickly fix the problem and in essence, prove their worth? What if you could provide
your boss, management team, or finance department with return on investment
(ROI) data and reports? Organizations are continuously looking for the best business
value, ways to justify spending and better yet, ways to reduce costs. And IT
departments are forced to look for software solutions that will reap the most benefit
from each dollar invested. Chances are if QA can prove automated testing can
positively affect your company’s bottom line, then QA would be viewed as a different
kind of hero – one that solves problems before they occur and cost the company
This RadView article outlines areas that QA can evaluate to gather ROI examples and
it offers ways to promote the value of testing and QA to your organization.
ROI Benefits of Testing
Finding & Fixing Bugs Early in the Development Lifecycle Saves Big Bucks
Industry studies have repeatedly shown that the earlier problems are found, the
more readily and cheaply they are solved. Scalability and functionality problems can
often lie at the heart of a web application, and as such may be difficult, time-
consuming, and costly to fix. Planning tests as the last check in an application’s
development potentially exposes you to tremendous risk and late changes, at the
very time you are trying to eliminate both. Leaving such tests to the end of the
development cycle leaves you very vulnerable.
What’s Your Hardware Capacity?
You can load test your web application’s hardware configuration in order to assess
capacity and optimize performance. By doing so, you may pleasantly learn you have
more hardware capacity available for testing than you thought. One RadView
customer, an international investment research firm, was planning to outsource the
testing of a new website to verify the performance and reliability of the application
and to determine the hardware requirements. They decided to perform the testing
in-house. Through the load testing process, they determined that its base hardware
estimate was sufficient to operate the website.
Testing for Memory Leaks
Just as a leaky faucet drains money from a homeowner, memory leaks on a web
application can drain time, energy, and resources from the technical staff.
Pinpointing and fixing memory leaks through testing can help avoid needless
technical tasks. For example, one RadView customer had a major financial website
that was experiencing instability during cookie verification in production. Through
load testing, the QA team discovered a memory leak. Prior to this discovery the
testing team was scheduling daily reboots of twenty-two of the site’s servers. With
the new testing results, the testing team was able to fix the instability issue and save
the company time and money.
ROI Benefits of Automating the Testing Process
Automated Testing Advantages
Testing finds problems. Automated testing finds problems sooner. What is it worth to
launch your web applications without problems and sooner? RadView has talked to
many customers about the advantages that automated testing provides over manual
testing. Some of these advantages include:
Overall total investment costs less with automated testing (script generation time
+ maintenance time + tools cost)
53% savings from reuse of scripts – savings from automated testing vs. manual
Testers spend 60% less time on script generation as a result of script reuse
Testers have more time available to analyze results and collaborate with
Development on investigating and reproducing bugs, which results in fixing them
sooner, and producing higher quality applications.
Script reuse throughout the various areas of testing – functional testing to
performance and stress testing – provides significant savings in time and money.
Everyone wants to increase productivity, but how do you measure this? Or how do
you calculate the cost savings realized through improved testing efficiency? Often
times this can be done by analyzing the testing cycles and more specifically the time
spent on writing scripts. Are you creating test scripts faster with an automated tool?
Are the tests more comprehensive and therefore, provide more test coverage? Does
your tool allow you to use the same script across different testing products? If the
answers to these questions are yes, then less time is spent on developing and coding
tests and more time is available to actually execute these tests and analyze the
results. In addition, having the ability to schedule tests to run, unattended, during
off hours is another way test teams can be more productive.
Choosing a Test Vendor that Provides the Best Value
Buy or License Only What You Need
When evaluating testing tools, ask the vendor about different licensing scenarios.
With the many different models of purchasing software today, you should be able to
purchase only what you need for your project. Are you interested in trying a
subscription license for one year? Will your testing needs last only 3 months? For
example, a California consulting firm was able to benefit from a RadView short-term
license to run load and stress tests for one of its major clients. Depending on your
project, you may not need long-term licensing.
Hardware and Space Requirements
Carefully consider the resource footprint of the test software as well as the ability of
the software to scale. Will you have to buy additional machines or memory to run the
test software? What’s the memory usage per virtual client? How much hardware will
you need to handle the load you need to simulate? Can you run test scripts on
multiple platforms and therefore, utilize the hardware you already own? One
RadView customer was able to cut hardware and memory needs in half by getting
more virtual users out of each PC. This is because the test software they chose had a
small footprint per virtual user and additional functionality for easy scalability.
Consider the Hidden Costs of Testing
How much time will a test engineer need for training? Will the engineer have to
spend a week away from the office to learn a new system or a proprietary scripting
language? Can they get up to speed and be productive quickly, or will it take a
month or so of time in the office, jammed in between other projects, which will slow
the start of the web application testing? Once testing is underway, will it require
constant rework to continue to run? Look for testing software that offers flexible
click recording to create and manage test scripts, in order to shorten the learning
curve, jump-start the testing process, and keep it moving.
RadView Software, Inc. RadView Software, Ltd.
7 New England Executive Park 14 Hamelacha Street
Burlington, MA 01803 Park Afek
Phone: 781-238-1111 Rosh Haayin, 48091 Israel
Fax: 781-238-8875 Phone: +972-3-9157060
Toll Free: 1-888-RADVIEW Fax: +972-3-9157683