A Test Generation Toolfor Specifications in theForm of State Machine       Zheng-Wen Shen         2004/11/23
References   A Test Generation Tool for Specifications in the    Form of State Machines     Q. M. Tan, A. Petrenko and G...
Outline1.        Introduction2.        Using the tool     1.      An Example     2.      Functions provided by the Tool   ...
1.        Introduction2.        Using the tool     1.      An Example     2.      Functions provided by the Tool     3.   ...
1. Introduction   TAG tool - Test Automatic Generation     Automatically generates test case for an FSM      specificati...
   FSM has been extensively used in the testing    phases of system developments.       E.g. conformance testing of comm...
Protocol conformance testing   A protocol specification generally can lead to    several implementations in software and/...
The existing test derivation methods   Check each transition in an FSM specification at    least once (branch coverage cr...
   Most of the existing protocols are not    completely specified.       Not all the sequences of interactions are fores...
1.        Introduction2.        Using the tool     1.      An Example     2.      Functions provided by the Tool     3.   ...
2. Using the tool   A complete test suite that guarantees full fault    coverage may be derived   A set of test cases th...
2.1 An example        To achieve a particular test purpose which is a certain         transition to be tested:    1.     ...
   State cover       The set of all preambles.   Transition cover       The set of sequences used to execute all speci...
   HSI method       Similar to the widely used W-method            A characterization set is used for state identificat...
The INRES responder
   Input alphabet       1- CR; 2- IDISreq; 3- ICONrsp; 4- DT0; 5- DT1   Output alphabet       0- NULL; 1- ICONind; 2- ...
Test suite and intermediate results   State Identifier = {{41}, {41}, {4}, {4}}   Preamble = {ε, 1, 13, 135}   Transiti...
2.2 Functions provided by the tool   A text file containing the FSM specification with    suffix “.fsm”     Symbol table...
   The test output is written in a text file.     Mnemonic format (.mnc)     SDL skeleton (.sdl)
2.3 the FSM specification1.   the state definitions2.   The input definitions3.   The output definitions4.   The transitio...
   The state definitions are a list of state names.   The input definition are a list of input names.   The output defi...
   The variable definitions define the variables in    parameters.       Integer, Charstring, Octetstring, Boolean   Us...
1.        Introduction2.        Using the tool     1.      An Example     2.      Functions provided by the Tool     3.   ...
3.1 Preambles   A tree with the initial state as its root is    constructed such that the tail states of the    outgoing ...
S0     1S1     3         5S2              S3         Preamble = {ε, 1, 13, 135 }
3.2 Postambles   A tree with given state as its root is constructed.   Once the initial state has been added to the tree...
S0S1          2S0     S2           S3              2        2     S0           S0       Preamble = {ε, 2, 2, 2 }
3.3 Transition Cover   The transition cover can be obtained by appending    each outgoing transition from this state to i...
3.4 State Identification Sequences.   The characterization set W is a set of input    sequence.   Harmonized state ident...
   For any two distinguishable states Si and Sj of S, there    exists a sequence σ that is a prefix of both σi ∈ Di and  ...
   How is Optimal ?    1.   The number of sequences in W is minimal.    2.   The sum of their lengths is minimal.   To o...
A heuristic solutionA1: A set P0 of all the state pairs that are    distinguishable for the given FSM is produced.A2: With...
The result of INRES responder   The result P0 of the algorithm A1 is the set of all    possible state pairs.   The chara...
The algorithm A1   The algorithm A1 is obtained by adapting the    FSM minimization algorithm give in    [ G. J. Holzmann...
The algorithm A2   Forms the root as the current node to probe   For each input word, a son is built if it can lead    t...
The search tree in A2
The algorithm A3   A sequence σk in W is selected such that    weighted sum of its length and the number of    the state ...
W = {σ0, σ1,… σn-1}  σk :                                              Remain state pairs   σ:                         dis...
1.        Introduction2.        Using the tool     1.      An Example     2.      Functions provided by the Tool     3.   ...
4. Experiments and ApplicationsState num.     10    50     100   150    200Transition num. 100 2500    10000 23500 40000CP...
1.        Introduction2.        Using the tool     1.      An Example     2.      Functions provided by the Tool     3.   ...
5. Conclusion   Proposed a heuristic solution to derive near-    optimal harmonized state identification sets.
Upcoming SlideShare
Loading in …5
×

20041113 A Test Generation Tool for Specifications in the Form of State Machine

526 views

Published on

A Test Generation Tool for Specifications in the Form of State Machine

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
526
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

20041113 A Test Generation Tool for Specifications in the Form of State Machine

  1. 1. A Test Generation Toolfor Specifications in theForm of State Machine Zheng-Wen Shen 2004/11/23
  2. 2. References A Test Generation Tool for Specifications in the Form of State Machines  Q. M. Tan, A. Petrenko and G. v. Bochmann  To appear in IEEE International Conference on Communications -- 96, Dallas, USA, 1996.
  3. 3. Outline1. Introduction2. Using the tool 1. An Example 2. Functions provided by the Tool 3. The FSM specification3. Test derivation methods 1. Preambles 2. Postambles 3. Transition Cover 4. State Identification Sequences4. Experiments and Applications5. Conclusion
  4. 4. 1. Introduction2. Using the tool 1. An Example 2. Functions provided by the Tool 3. The FSM specification3. Test derivation methods 1. Preambles 2. Postambles 3. Transition Cover 4. State Identification Sequences4. Experiments and Applications5. Conclusion
  5. 5. 1. Introduction TAG tool - Test Automatic Generation  Automatically generates test case for an FSM specification.  Transition identification approach for test derivation  Output test cases in the form of an SDL skeleton.
  6. 6.  FSM has been extensively used in the testing phases of system developments.  E.g. conformance testing of communication protocols. Communication protocols are the rules that govern the communication between the different components within a distributes system.  7-layer OSI model
  7. 7. Protocol conformance testing A protocol specification generally can lead to several implementations in software and/or hardware. Assure the compatibility with other implementations of the same protocol.
  8. 8. The existing test derivation methods Check each transition in an FSM specification at least once (branch coverage criteria.) Verify the tail state of the transition to obtain high fault coverage and to guarantee conformance in the context of a more general fault model.  Using state identification techniques  Defect the fault that the FSM enters a different state than specified.
  9. 9.  Most of the existing protocols are not completely specified.  Not all the sequences of interactions are foreseen. The existing test derivation methods for protocols are limited to completely specified specifications. TAG working directly for deterministic, partially specified FSM specifications.
  10. 10. 1. Introduction2. Using the tool 1. An Example 2. Functions provided by the Tool 3. The FSM specification3. Test derivation methods 1. Preambles 2. Postambles 3. Transition Cover 4. State Identification Sequences4. Experiments and Applications5. Conclusion
  11. 11. 2. Using the tool A complete test suite that guarantees full fault coverage may be derived A set of test cases that cover a given test purpose may be derived. Output format  Mnemonic format  SDL skeleton
  12. 12. 2.1 An example To achieve a particular test purpose which is a certain transition to be tested: 1. Bring the FSM from its initial state to the starting state of the transition under test using the shortest input sequence possible. – a preamble of the test case 2. Execute the transition and check the observed output 3. Check a tail state of the transition by observing its reaction to a pre-selected set of state identification sequences. 4. Apply an input sequence to return to the initial state of the FSM. – a postamble of the test case
  13. 13.  State cover  The set of all preambles. Transition cover  The set of sequences used to execute all specified transitions. State identification sequences  Distinguish states by their output reactions.  TAG implements the HSI method
  14. 14.  HSI method  Similar to the widely used W-method  A characterization set is used for state identification.  A tuple of subsets of a W set.  Can be applied to partially specified FSMs.
  15. 15. The INRES responder
  16. 16.  Input alphabet  1- CR; 2- IDISreq; 3- ICONrsp; 4- DT0; 5- DT1 Output alphabet  0- NULL; 1- ICONind; 2- DR; 3- CC; 4- ACK0; 5- ACK0,IDATind; 6- ACK1; 7- ACK1,IDATind States  S0- Closed (the initial state); S1- Opening; S2- Waiting_DT0; S3- Waiting_DT1
  17. 17. Test suite and intermediate results State Identifier = {{41}, {41}, {4}, {4}} Preamble = {ε, 1, 13, 135} Transition Cover = {1, 4, 5, 11, 12, 13, 14, 15, 131, 132, 134, 135, 1351, 1352, 1354, 1355} Postamble = {ε, 2, 2, 2} Test Suite = {41, 441, 541, 1241, 13241, 135241, 141, 1141, 1441, 1541, 134, 1314, 13514, 13514, 13554}
  18. 18. 2.2 Functions provided by the tool A text file containing the FSM specification with suffix “.fsm”  Symbol table (.tbl)  FSM structure (.cpl) Complete test derivation  A complete test suite is generated. Selective test derivation  Give the test purpose, which specifies one transition in the FSM.
  19. 19.  The test output is written in a text file.  Mnemonic format (.mnc)  SDL skeleton (.sdl)
  20. 20. 2.3 the FSM specification1. the state definitions2. The input definitions3. The output definitions4. The transition definitions5. The variable declaration (optional)6. The homing sequence definition (optional)7. “end;”
  21. 21.  The state definitions are a list of state names. The input definition are a list of input names. The output definitions are list of output names. The transition definitions define the FSM state table itself by a list of transition specifications.  Current state name, input name, output name and next state name.  May be followed by a set of the comments.
  22. 22.  The variable definitions define the variables in parameters.  Integer, Charstring, Octetstring, Boolean Using Keyword “homing” to give a sequence of input names as a homing sequence.
  23. 23. 1. Introduction2. Using the tool 1. An Example 2. Functions provided by the Tool 3. The FSM specification3. Test derivation methods 1. Preambles 2. Postambles 3. Transition Cover 4. State Identification Sequences4. Experiments and Applications5. Conclusion
  24. 24. 3.1 Preambles A tree with the initial state as its root is constructed such that the tail states of the outgoing transitions from the state corresponding to a current node. All nodes in this tree must become a current node once and only once in the order that they enter this tree. The path from the root to a given node is preamble for the corresponding state.
  25. 25. S0 1S1 3 5S2 S3 Preamble = {ε, 1, 13, 135 }
  26. 26. 3.2 Postambles A tree with given state as its root is constructed. Once the initial state has been added to the tree, the procedure stops. The path from the root to the last added node is a postamble from the given state. Maybe no postamble from the given state.
  27. 27. S0S1 2S0 S2 S3 2 2 S0 S0 Preamble = {ε, 2, 2, 2 }
  28. 28. 3.3 Transition Cover The transition cover can be obtained by appending each outgoing transition from this state to its preamble. Preamble = {ε, 1, 13, 135 }  S0:ε::1, ε::4, ε::5  S1: 1::1, 1::2, 1::3, 1::4, 1::5  S2: 13::1, 13::2, 13::4, 13::5  S3: 135::1, 135::2, 135::4, 135::5 Transition Cover = {1, 4, 5, 11, 12, 13, 14, 15, 131, 132, 134, 135, 1351, 1352, 1354, 1355}
  29. 29. 3.4 State Identification Sequences. The characterization set W is a set of input sequence. Harmonized state identification sets (HSI sets), subsets of W.  {D0, D1, …, Dn-1}  Di is a set of prefixes of sequences in W  n is the number of states of S.
  30. 30.  For any two distinguishable states Si and Sj of S, there exists a sequence σ that is a prefix of both σi ∈ Di and σj ∈ Dj such that σ can be accepted by these two states and produces different outputs.
  31. 31.  How is Optimal ? 1. The number of sequences in W is minimal. 2. The sum of their lengths is minimal. To obtain a minimal characterization set for a give FSM maybe an NP-Hard problem.
  32. 32. A heuristic solutionA1: A set P0 of all the state pairs that are distinguishable for the given FSM is produced.A2: With P0, a characterization set W is obtained by forming a search tree from the input alphabet to distinguish the state pairs in P0.A3: From this W, some HSI set Di is selected for each state i, such that the number of distinguishable state pairs and the length of a sequence are traded off.
  33. 33. The result of INRES responder The result P0 of the algorithm A1 is the set of all possible state pairs. The characterization set W from the algorithm A2 is {41} The HSI sets {{41}, {41}, {4}, {4}}
  34. 34. The algorithm A1 The algorithm A1 is obtained by adapting the FSM minimization algorithm give in [ G. J. Holzmann. Design and Validation of Computer Protocols, 1991 ]
  35. 35. The algorithm A2 Forms the root as the current node to probe For each input word, a son is built if it can lead to a better solution than other build nodes. If it is estimated that a subtree with a minimal number of branches and with a minimal average length that distinguishes a maximal number of state pairs could be formed by probing an unprobed son of ti.
  36. 36. The search tree in A2
  37. 37. The algorithm A3 A sequence σk in W is selected such that weighted sum of its length and the number of the state pairs that it can not distinguish in the left state pairs is minimal. For each remaining state pair (l, m) that can be distinguished by σk, find a prefix of σk such that (l, m) is distinguished. Then the prefix is put into the HSI sets Dl and Dm.
  38. 38. W = {σ0, σ1,… σn-1} σk : Remain state pairs σ: distinguish ? (l, m)HSI Sets: {D0, D1, D2, .. Dl, …, Dm, …Dn-1}
  39. 39. 1. Introduction2. Using the tool 1. An Example 2. Functions provided by the Tool 3. The FSM specification3. Test derivation methods 1. Preambles 2. Postambles 3. Transition Cover 4. State Identification Sequences4. Experiments and Applications5. Conclusion
  40. 40. 4. Experiments and ApplicationsState num. 10 50 100 150 200Transition num. 100 2500 10000 23500 40000CPU time (sec.) 0.11 2.96 40.87 197.06 760.89
  41. 41. 1. Introduction2. Using the tool 1. An Example 2. Functions provided by the Tool 3. The FSM specification3. Test derivation methods 1. Preambles 2. Postambles 3. Transition Cover 4. State Identification Sequences4. Experiments and Applications5. Conclusion
  42. 42. 5. Conclusion Proposed a heuristic solution to derive near- optimal harmonized state identification sets.

×