QC Sprinter Whitepaper

1. How to increase the efficiency of
manual testing
Business white paper

2. Table of contents
Executive summary ..................................................3
Introduction ............................................................3
The real challenges of manual testing .........................5
Reporting a defect ...................................................7
Data injection .........................................................8
Mirror Testing .........................................................9
Reporting ............................................................. 10
Conclusion ............................................................ 11

3. Even though nearly Executive summary Introduction
every QA team in Manual testing remains the most popular Application testing has been around for
the world still relies
on manual testing, method for validating the functionality decades. Ever since the first software
the actual process of
manual testing hasn’t
of software applications. It is simple, applications were built, there was a need
improved much in the straightforward, and doesn’t require to validate an application’s functionality
past 10 or even 20
years. advanced technical knowledge or before releasing it into production. The
programming skills. However, as original validation method was manual
technologies have progressed and testing. A software tester—or sometimes a
applications become more complex, business user—through a series of steps,
the process of manual testing has stayed would interact with an application and
mostly unchanged. It is still a very document any problems encountered along
resource-intensive and often repetitive effort the way. Naturally, this process was often
that requires monotonous reiteration of monotonous, time consuming, and
the test steps and manually entering large error prone.
sets of data. Manual testing can also be Advances in technology brought new
highly error prone, resulting in overlooked ways of application testing, including
defects and false positives. Over the years, automated functional testing. At first glance,
several solutions have been introduced to automation looked like the answer to all of
the market to help run manual tests, but the problems with manual testing. It didn’t
they still failed to solve the fundamental rely on a business user to execute repetitive
problems that plague most manual testing test steps; it was scalable and reusable;
teams: data-driven testing, accurate defect and it could run tests all night, unattended,
reporting, better communication with and test different business processes with
developers, and replaying the same tests in various sets of data. However, QA teams
multiple environments. quickly realized that not all applications
This paper explores the challenges of are good candidates for automated testing,
traditional manual testing methods and and that the best testing strategies always
discusses the advantages of HP Sprinter— involved a combination of automated and
a unique solution for maximizing the manual testing elements.
efficiency and precision of manual testing, Manual testing is here to stay. There
streamlining communication between testing will always be applications or parts of
and development teams, and reducing applications that should be tested manually
test cycles. by a business user who is familiar with
3

4. Figure 1: HP Sprinter features an easy-to-use, unobtrusive interface
the business process flow. And there will Most importantly, traditional manual runners
always be times in the application lifecycle don’t solve the main problems that plague
where quick manual exploratory testing manual testing: data entry, inaccurate and
will provide just enough information for inadequate defect reporting, and replaying
developers to capture early defects, before tests with multiple configurations.
they turn into larger problems downstream. HP sets out to change the way QA
But even though nearly every QA team organizations view manual testing.
in the world still relies on manual testing, HP Sprinter software is a truly revolutionary
the actual process of manual testing hasn’t approach to all of the aspects of manual
improved much in the past 10 or even 20 testing: from maximizing the “real estate”
years. It is still a very resource-intensive, on your screen for easier interaction with an
time-consuming, and often tedious effort, application, to accelerating the capturing
which involves endless repetitions of the and reporting process of relevant defects,
same steps, and copying and pasting injecting test data, and even replicating
countless sets of data. Several test test steps across multiple environments on
management tool vendors have offered remote machines. Intuitive and easy to use,
manual test runners in the past, but business HP Sprinter is one of the only solutions
users are reluctant to accept them. Most that helps preserve the benefits of manual
testers still prefer to print the test steps testing, while significantly improving its
from the tool’s repository and continue to efficiency, accuracy, and speed.
run tests in the old way, often losing the
traceability of the test execution results.
4

5. Figure 2: HP Sprinter offers multiple ways to log defects to give developers detailed
information about the issue
Test data entry can The real challenges of manual testing application’s functionality with different sets
now be completed in The advantages of manual testing are of data—such as login names or credit
a matter of seconds,
significantly reducing undeniable. It doesn’t involve complex card numbers—testers usually print out
test execution time and
helping organizations
preparations, script recording, or coding. a spreadsheet with available data and
release applications It can be performed by a business then manually enter this information into
faster.
analyst or any other application end user an application. Alternatively, they may
without requiring any particular technical toggle between an open spreadsheet and
knowledge, specialized tools, or intricate the application under test, copying and
setup. Manual testing can be run at pasting data into required fields. But when
any stage of an application lifecycle, schedules get tighter and testers are under
making it even more popular with agile pressure to finish validating the application’s
development teams. functionality before the release, they often
resort to entering bogus data into form
But traditional manual testing has some
fields, missing potentially critical problems.
very serious drawbacks as well. Not
only is it monotonous and repetitive, it It is also extremely difficult to achieve good
can also be highly error prone, where a coverage of different environments and
missed step or incorrect data can lead platforms when testing manually. Testers
to overlooked defects and false results. simply don’t have the time to repeat the
Data-driven manual testing also presents same steps over and over, on different
a considerable challenge: To validate the machines running various combinations of
5

6. Figure 3: Data injection helps to save a significant amount of time and helps to reduce errors
“Sprinter’s Mirror browsers and operating systems, to look for exploratory testing, where testers “poke
capability allows for differences in application behavior under around” in an early build of an application
much faster (>80%
productivity) Web diverse conditions. QA managers end up without following any formal written test
application Browser
Compatibility testing”
choosing a single—and typically the most steps or procedures. If a defect is found,
Anonymous customer,
common—platform to test the application testers often cannot even recall which path
TechValidate survey, on, ignoring other possible combinations they took through an application to arrive at
October 2010
and use cases. that point. Naturally, recreating such defects
is a challenging task for R&D.
Finally, when defects are found, manual
testers typically don’t have the tools or skills Introducing HP Sprinter: a revolutionary
to properly communicate the problem. As a approach to manual testing
result, the developer who is investigating a HP Sprinter is a unique, innovative
particular defect has to try and reproduce solution designed to increase efficiency,
it with only minimal information. If the coverage, and precision of manual testing.
developer knew which steps the tester A component of HP Application Lifecycle
took before arriving at a problem, which Management (ALM) and HP Quality
conditions the application was tested under, Center (QC), HP Sprinter is an
or what data was entered, the resolution easy-to-use manual testing tool that
time could have been much faster. Instead, features an intuitive, unobtrusive interface
developers often have to rely on poorly and a series of advanced features to help
constructed and incomplete defect records, testers improve testing speed, accuracy,
which cause unnecessary delays and and documentation. With HP Sprinter,
frustrations. This is especially true in agile testers no longer need to flip between
6

7. Figure 4: Mirroring allows you to simultaneously test multiple configurations, saving time
and increasing coverage
multiple screens looking for test-case Reporting a defect
descriptions. HP Sprinter can run as a When creating a defect report, manual
collection of sidebars on the screen, testers typically don’t like to spend time
which can be expanded or minimized typing up large amounts of data to help
as needed. It can also show test steps as developers reproduce the issue. HP Sprinter
one-line subtitles, maximizing the screen solves this problem by automatically
“real estate” needed for interacting with the including an array of relevant information
application under test. in the defect description, to help illustrate
HP Sprinter runs on your local computer. the issue and the actions that led up to
And, because it interacts with HP ALM uncovering it. HP Sprinter allows you to
software and HP QC software, it has full either submit a defect into the HP ALM or
access to the tests that have been defined HP QC repository during the test run, or
in your test plan, as well as complete to continue with the run flow and submit
information about the application under a defect when the run is completed. All of
test. When you first open a test in the information about steps, user action
HP Sprinter, you can select which screenshots, and detailed descriptions
application you will be testing and review that were available during the run is also
its configuration. HP Sprinter automatically accessible when the run is finished.
starts the application or browser, and
opens the application under test.
7

8. The tester can configure which types of amount of relevant defect information
information HP Sprinter is going to include without slowing down the testing process
into the defect report. You can either submit or creating any additional steps. All of
a Smart Defect that would automatically the defect data can help testers to better
include all of the selected data, or open communicate with application developers,
a new defect dialog with no predefined and to save the developer’s time in
content. To help illustrate the problem, understanding, reproducing, and resolving
you can include a detailed list of user the problem. Defects submitted through
actions—including every button click and HP Sprinter are automatically linked to the
piece of data entered—that you took prior test run and to any associated requirements
to discovering a defect. You can add a in HP ALM or HP QC, and therefore create
short movie replaying all of the user actions traceability and support the concept of
and application responses over the past requirements-driven testing.
several minutes. You can also choose to
Data injection
include a screen capture, complete with
Most test scenarios require testers to fill
annotations and markups. The annotations
out long forms, such as entering shipping
feature is also very useful when testing the
information or submitting credit card
application’s UI elements. For instance,
numbers. Data entry is indeed one of
you can measure the length of a form
the most time-consuming, tedious, and
field using a measuring tool in the
error-prone activities in manual testing.
annotations workspace.
HP Sprinter automates test data entry by
The ability to attach user actions and a using the process of data injection, saving
movie of all recent activities is invaluable valuable testing time, increasing coverage,
in exploratory testing and testing in agile and reducing errors. Data sets can
environments. Not only can a developer be prepared in advance, and can be
easily recreate the problem, the list of steps added from the file system or from the
can be used later to create a more formal Test Resources module in HP ALM or
test script. HP QC. Data sets can be associated with
The intuitive HP Sprinter UI and smart the application under test, and can be
settings allow testers to submit a great used directly by HP Sprinter. Data files can
be either in Microsoft® Excel or Comma
8

9. Separated Values (.csv) format, with the With HP Sprinter, you can execute tests
column headers set up to match the form on your primary machine—such as your
fields in the application under test. local laptop or desktop—and HP Sprinter
automatically repeats the exact same test
Data injection automatically injects
on different secondary machines. This
data from the row in the data set into
feature is known as Mirror Testing. It allows
the matching fields in the application,
you to replicate all of the actions that you
populating the form fields in a fraction of
take on your primary machine on multiple
the time it would have taken to manually
computers with different configurations.
enter the data. In the next iteration of
This distinctive capability allows QA teams
the test, you can choose values from the
to instantly increase test coverage without
next row, validating the application with
having to spend the time and resources
multiple sets of data. You can also control
manually repeating tests on different
the order of the data injection, which is
browser and OS combinations.
especially beneficial in testing applications
with dynamic fields. This process, which HP Sprinter automatically shows you the list
used to take manual testers several minutes of available machines that are connected
for each form, can now be completed in a to your primary computer. You can choose,
matter of seconds, significantly reducing testfor example, to have one machine set up
execution time and helping organizations to test the application in Internet Explorer
release applications faster. (IE) 7.0 and another machine set up to test
the application in Mozilla Firefox or IE 8.0.
Mirror Testing
HP Sprinter connects to and initializes all of
One of the most significant limitations of
the selected machines, and then starts the
manual testing is that there’s never enough
application and continuously displays health
time to run tests on multiple machines, using
indicators and run status for each machine.
different configurations and platforms.
Each mouse click, field entry, or page
9

10. advance that you perform on your primary Reporting
machine is automatically replicated on each When the run ends, it is automatically
of the secondary machines. All actions are updated in HP ALM and HP QC, and a
kept in sync, and HP Sprinter compares detailed run report is created. The report
the results on all of the computers that are includes the run summary and all of the
running the test. To enhance performance, steps that were performed, including actual
HP Sprinter does not compare results results that were entered during the run,
automatically after each user action, but and screen captures that were added. You
can be configured to compare results on can also see a list of any defects that were
demand at critical points in the test. As submitted during the run. Alternatively,
soon as HP Sprinter detects a difference, you can review the run results through
you can view and compare the machines, the Storyboard view. In the Storyboard
and capture screenshots of all of the view, the run is represented in a series of
machines that display inconsistent results. screen captures that were taken for every
From there, you can either submit a defect user action that was performed during the
and automatically include the description run. The report includes a list of all user
of the difference, or instruct HP Sprinter to actions that were performed during the test
ignore the difference. Once the “Ignore” run, making it easier to create a format
rule is created, HP Sprinter automatically test from an informal or exploratory test.
recompares the machines. The run report is also saved in HP ALM,
which helps to keep all of the members of
Today’s applications are required to run
the testing and development teams on the
in a variety of environments and support
same page with the status and progress
different types and versions of browsers,
of the testing process, and helps them to
operating systems, and configurations.
make informed decisions about whether the
Without HP Sprinter, it is nearly impossible
application is ready to be released.
to manually test all of the required
combinations. The ground-breaking
mirroring technology not only reduces
time, but helps to increase testing
coverage and to release applications
with more confidence.
10

11. Conclusion defect reporting, test run mirroring, and
Manual testing continues to be a popular other superior features, HP Sprinter is
method for validating application changing the way QA teams view
functionality. To help accelerate the manual manual testing. Instead of a tedious,
testing process and to increase coverage, time-consuming, and error-prone process,
speed, and accuracy, HP Sprinter provides manual testing with HP Sprinter can be
an easy-to-use, unobtrusive tool that is accurate, fast, and complete, helping to
seamlessly integrated with HP ALM and increase application quality, reduce testing
HP QC. HP Sprinter features an intuitive time, and improve communication between
UI that helps to maximize the screen “real different stakeholders in the organization.
estate” of the application under test, and
a variety of powerful advanced features
designed to remove the major inefficiencies
and difficulties of the manual testing
process. Through data injection, detailed
Improve testing speed and accuracy with HP Sprinter; visit www.hp.com/go/sprinter
11

12. Share with colleagues
Get connected
www.hp.com/go/getconnected
Get the insider view on tech trends, alerts, and
HP solutions for better business outcomes
© Copyright 2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only
warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein
should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
Microsoft is a U.S. registered trademark of Microsoft Corporation.
4AA3-1859ENW, Created November 2010