EuroSTAR Software Testing Conference 2009 presentation on A Sneaky Way to Introduce More Automated Testing by Seretta Gamba. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

1. A Sneaky Way to Introduce More Automated
Testing
W11
Stockholm 02.12.2009

2. Agenda.
 The Problem.
 The “sneaky” Idea.
 The Realization.
 Conclusion and general discussion.
A Sneaky Way to Introduce More Automated
Testing

3. Chapter 1.
The Problem.
 The company.
 The application.
 The developer team.
Overview:
1 The Problem.
2 The Idea.
3 The Realization.
4 Conclusion.
1

4. The Company
Steria Mummert Consulting is
one of the top ten IT companies
in Germany (Counseling, IT-
Service). In Germany and Austria
about 1700 employees
Mummert ISS GmbH was established
2002 as a 100% subsidiary of
Mummert Consulting. Develops
standard software for insurance
companies. About 50 employees
Steria
Mummert
ISS
Steria Mummert Consulting
Steria Group
Steria Mummert ISS is a small and successful company that develops standard
software for insurances
1
XXXXXXXX.The Problem.
The Steria Group is one of
the top ten IT-Service
companies in Europe with
worldwide about 18.000
employees.

5. The Application
The product is:
 a complete software solution for insurance companies,
 a standard software, that enables a flexible management of the
products and processes of an insurance company and with
which both partner and portfolio can easily be maintained.
System design:
 Completely newly developed standard solution.
 Customer oriented application structure.
 …..
1
Funktionsumfan.The Problem.
The application has a standardized core and offers great flexibility in modeling the
individual processes of an insurance company.

6. The Developer Team
Test
Automation
Team
New Customer
New Customer
New Customer
Product team
1
XXXXXXXX.
Some years ago the product team was able to support both new customers and the
automation team, currently it has trouble to support even only the customers.
The Problem.
New Customer
New Customer
New Customer
X
Old Customer
Old Customer
Old Customer

7. Chapter 2.
The “sneaky” Idea.
 What do we need in order to be able to automate?
 Can manual testing deliver it?
 Can we convince our colleagues to switch over to the
“automation” framework for manual testing?
 Can we extract and use the collected Information for
automation?
Overview:
1 The Problem.
2 The Idea.
3 The Realization.
4 Conclusion.
2

8. Prerequisites for automation2
Funktionsumfan.The Idea.
Testers that execute manual tests have the information needed to automate those
same tests
By performing manual tests a tester can deliver at least:
Knowledge of the application under test.
Test specifications.
What do we need in order to be able to automate?
 Knowledge of the application under test.
 Test specifications.
 Automation strategy.
 Test-Tools.

9. Automation Strategy: Command-Driven Testing
 Step 1
 Step 3
 Step 2
Prerequisites
2
XXXXXXXX.
Capture
Tool
Script
Tool
Script
ISS
ISS Test Station
Driver Data
Driver
Data-
Template
Suite
Test cases
Registration of
Gui Controls
 1.
 3.
Our Command-Driven framework supports the test automation team by (mostly)
eliminating the need to program new keywords and by automating some of the steps
Automated test
Capture-Replay
Tool
Suite
Test results
 2.
Command Library
The Idea.
Tester
Tester

10. Supporting manual testing2
Funktionsumfan.The Idea.
The original testing procedure was based on excel-test-sheets, that would be
performed again and again for regression testing.
Original testing process
 Delete old results from the template.
 Enter eventual corrections, improvements or just plain new test cases.
 Enter the new results.
 Calculate metrics.
 If defects are found, create a defect-item in the development control system.

11. Weaknesses of the manual procedure2
XXXXXXXX.
Test case maintenance
3. Cumbersome defect
reporting.
4. Test cases tend to be
repeated in order to
report the defects
exactly.
Defect reporting
5. No automatic statistic of
the executed test cases
versus the defects
found.
Summary reportingSummary reporting
In order to support manual testing with the automation framework, we had to find out
how testers worked and devise better support.
1. Corrections or
improvements to the test
template can get lost.
2. Test-specifications are
very “concise”.
The Idea.

12. Available features in the automation framework2
XXXXXXXX.
Add or change test cases.
Overview of the available test cases.
Logging of test results.
Result reports or overviews.
Prioritization of the test cases.
Defect reporting.
Availablefeatures
Must have Enhancement
X
X
X 1
X 5
X
X 3,4
The Idea.
Many of the available features in the automation framework could, with no change,
be employed to support manual testing.

13. Features that have to be implemented2
.
New features had to be implemented in the automation framework in order to be able
to support manual testing adequately .
 Test driven by sequential
display of the selected
test cases.
 Paperless execution.
 Support in:
Support for manual test
execution
 Import from the current excel sheets (csv-files) to the test suites in the framework format.
 Import functionality driven by an external table.
Import test-template
sheets
 Capture-facilityintegrated into the framework.
 Generationof a capture-replayscript with simultaneous recording.
 Starts or stops bypushing a button.
 Recording automatically attached to defect tracking item.
Detail test execution for
developers
 Integration of SQL-scripts to extract data from the database.
Save application status
to file
The Idea.
 Performing tests.
 Maintaining test
cases.
 Creating defect
items.
 Test recording.

14. Chapter 3.
Overview:
1 The problem.
2 The Idea.
3 The Realization.
4 Conclusion.
3The Realization.
 Manual testing with the automation framework.
 Automating the manual tests.

15. …
Manual test execution with the automation framework3
Test execution is driven by the framework, but the tester is free to decide what she
wants to do next and how
Set preconditions
and export them to file
Start recording
Perform test case
Export post-
conditions to file
Check
test case
display test
case
OK
Defect
The Realization.
Start test suite

16. Automating the manual tests
 Step 1
 Step 3
 Step 2
Prerequisites
Capture
Tool
Script
Tool
Script
ISS
ISS Test Station
Driver Data
Driver
Data-
Template
Suite
Registration of
Gui Controls
 1.
 3.
After the testers started working with the framework, creating more automated tests
has been much more rewarding because some steps have already been performed
Automated test
Capture-Replay
Tool
Suite
Test results
 2.
Command Library
Test cases
1
2
1
3
.The Realization.

17. Automating the manual tests3
 Call the framework functionality to transform the captured tool-scripts into Command-Driven scripts.
Step 1
 Rework the command-Driven scripts.
Step 2
 Create the new test procedures.
Step 3
Insert the test procedures to perform the test cases in the test suite structures.
Step 4
The input from manual testing reduced the effort to automate a test case to the
following steps
The Realization.
Check the automated test suite

18. Chapter 4.
Overview:
1 The problem.
2 The Idea.
3 The Realization.
4 Conclusion.
4Conclusion.
 Short summary of our experiences.
 Next steps.
 General discussion.

19. Short summary of our experiences4
XXXXXXXX.
After some time to allow the testers to get acquainted with the new process, we
started to get the first results.
Conclusion.
 The testers had a hard time getting used to the new formats.
 The import facility, test-sheet  test suite was well accepted from the
very beginning.
 The recording feature was very well received by both testers and
developers.
 In the beginning the testers were recording either too much or too little.
 Testers from other product families also started to use the framework.

20. Next4
XXXXXXXX.
…and they lived happily ever after. Well, we still have lots of effort and work to do.
Conclusion.
 Partially automated tests.
 Sophisticated metrics.
 Integration with test planning and management tools .

21. It’s better to light a candle than to curse darkness
Confucius