Głównym wyzwaniem w walidacji oprogramowania jest zaprojektowanie testów, tak aby obejmowały one wszystkie wymagania. Prezentacja zawiera opis wypracowanych metod, które znacznie poprawiły proces projektowania testów w zespole walidacji, zmniejszając ilość pracy, a jednocześnie zwiększając wydajność i jakość zaprojektowanych testów.

1. TEST DESIGN TUTORIAL
CZYLI SZTUKA PROJEKTOWANIA TESTÓW
AGNIESZKA GARWOLIŃSKA
7.06.2017

2. INTRODUCTION
The main challenge in software validation is to design tests
so they cover all requirements keeping right balance. It
means to limit quantity of tests, at the same time ensuring
enough use cases are verified.
COST
RISK Optimum security
at minimum cost

3. INTRODUCTION
This tutorial provides Best Known Methods that
significantly improved my test design process by
reducing the workload and simultaneously increasing
productivity and the quality of designed tests.
The method based on the discipline in the
implementation of the test phases and using
standardized test architecture.

4. THREE FUNDAMETAL STEPS
 Recognizing WHAT?
 Architecture HOW?
 Utilities HOW?
REQUIREMENTS FEATURE SPECIALIST CONSULTATION
OBJECTIVE PRECONDITIONS
TEST
PREPARATION
TEST
RUN
TEST
CLEAN-UP
TOOLS DICTIONARIES CODING STANDARDS

5. RECOGNIZING:
WHAT I NEED TO TEST?
You need to find out how should it work:
 Which requirements describes tested feature
 How this feature usually works
 What are the weaknesses of this feature
Example:
Requirement: Linux driver must cooperate with native kernel SW
RAID solution
SW RAID 0 SW RAID 1
SW RAID 2 SW RAID 3SW RAID 4
SW RAID 5
SW RAID 6
SW RAID 1 + 0
WORKS PROPERLY...
ARCHITECTURERECOGNIZING UTILITIES

6. SW RAID 3
SW RAID 0 SW RAID 1
SW RAID 2SW RAID 4
SW RAID 5
SW RAID 6
SW RAID 1 + 0
WORKS PROPERLY...
ARCHITECTURERECOGNIZING UTILITIES
You need to find out how should it work:
 Which requirements describes tested feature
 How this feature usually works
 What are the weaknesses of this feature
Example:
Requirement: Linux driver must cooperate with native kernel SW
RAID solution
RECOGNIZING:
WHAT I NEED TO TEST?

7. You need to find out how should it work:
 Which requirements describes tested feature
 How this feature usually works
 What are the weaknesses of this feature
Example:
Requirement: Linux driver must cooperate with native kernel SW
RAID solution
SW RAID 0 SW RAID 1 SW RAID 5 SW RAID 1 + 0
WORKS PROPERLY STANDARD
CONDITIONS
AFTER RAID
RESTART
AFTER DUT
REBOOT
AFTER DUT
SHUTDOWN
ARCHITECTURERECOGNIZING UTILITIES
RECOGNIZING:
WHAT I NEED TO TEST?

8. A great solution is to build a test matrix arising from
requirements analysis. This matrix must capture the
different possibilities, features, and combinations of
factors that affect the behavior of our tested object.
In this way it is possible to easily determine the required
test cases, their scope, and quantity.
The test cases are created in an orderly, systematic way,
keeping strictly considered test architecture.
ARCHITECTURERECOGNIZING UTILITIES
ARCHITECTURE OF TEST DESIGN

9. ARCHITECTURERECOGNIZING UTILITIES
HOW SHOULD I BUILD TEST SUITE?
TEST
MATRIX
Mode X
on: 1
off: 0
Special
conditions
RAID 0 RAID 1 RAID 5 RAID 1+0
Verify for 1
device
0 TC_R0_ON1_01 TC_R1_ON1_01 n/a n/a
1 TC_R0_ON1_02 TC_R1_ON1_02 n/a n/a
1 RAID restart TC_R0_ON1_03 TC_R1_ON1_03 n/a n/a
1 Reboot DUT TC_R0_ON1_04 TC_R1_ON1_04 n/a n/a
1 Shutdown TC_R0_ON1_05 TC_R1_ON1_05 n/a n/a
Verify for 2
devices
0 TC_R0_ON2_01 TC_R1_ON2_01 TC_R5_ON2_01 n/a
1 TC_R0_ON2_02 TC_R1_ON2_02 TC_R5_ON2_02 n/a
1 RAID restart TC_R0_ON2_03 TC_R1_ON2_03 TC_R5_ON2_03 n/a
1 Reboot DUT TC_R0_ON2_04 TC_R1_ON2_04 TC_R5_ON2_04 n/a
1 Shutdown TC_R0_ON2_05 TC_R1_ON2_05 TC_R5_ON2_05 n/a
...

10. HOW SHOULD I FORMULATE
TEST OBJECTIVE?
Verify if SW RAID [0, 1, 5, 10] works properly embedded on [1, 2, 3, 4, 5, 6]
[Device 1, Device 2] with Mode X [on, off] works properly [ , after RAID restart, after reboot
DUT, after DUT shutdown]
ARCHITECTURERECOGNIZING UTILITIES
TEST
MATRIX
Mode X
on: 1
off: 0
Special
conditions
RAID 0 RAID 1 RAID 5 RAID 1+0
Verify for 1
device
0 TC_R0_ON1_01 TC_R1_ON1_01 n/a n/a
1 TC_R0_ON1_02 TC_R1_ON1_02 n/a n/a
1 RAID restart TC_R0_ON1_03 TC_R1_ON1_03 n/a n/a
1 Reboot DUT TC_R0_ON1_04 TC_R1_ON1_04 n/a n/a
1 Shutdown DUT TC_R0_ON1_05 TC_R1_ON1_05 n/a n/a
...

11. Objectives are defined.
What Preconditions must be fullfiled in order to be able
run the test?
Which of them are repetitive?
Which are rather unique?
What is main scope of the test?
What needs to be clean up after all?
HOW SHOULD I DESIGN TEST CASE?
ARCHITECTURERECOGNIZING UTILITIES
OBJECTIVE PRECONDITIONS
TEST
PREPARATION
TEST
RUN
TEST
CLEAN-UP
PRECONDITIONS
PRECONDITIONS
TEST PREPARATION
TEST RUN
TEST CLEAN-UP
TEST
PREPARATION
TEST
RUN
TEST
CLEAN-UP

12. ARCHITECTURERECOGNIZING UTILITIES
OBJECTIVE PRECONDITIONS
TEST
PREPARATION
TEST
RUN
TEST
CLEAN-UP
TOOLS DICTIONARIES CODING STANDARDS
EXCEL ALM PRECONDITIONS
PASS/FAIL CRITERIA
PYGUIDEPEP 0008 GVP
HOW SHOULD I DESIGN TEST CASE?
UTILITIES

13. DZIĘKUJĘ ZA UWAGĘ 