This document discusses methods for creating exhaustive test sequences to fully test devices. It proposes representing all possible test sequences as paths in a graph where vertices are individual "test atoms" and edges represent allowed sequential execution. This allows generating all admissible paths as a set of complete test sequences. It suggests using multithreading to run tests with varying time delays, improving coverage. While full coverage is impossible to achieve, prioritizing high risk test atoms can generate focused test sets that find more failures than traditional methods. Open questions remain about optimizing test selection strategies.

1. Stars theory
Aplit-soft present

2. What’s a minimal test mean?
• Simplex and complex test
• “WiFi On “ is minimal
• File transfer through BT isn't minimal:
• Power on
• Pairing
• Connection
• Transfer
• Two kinds of tests
• Independent
• Depend on previous step

3. Test flow diagram
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
file
BT receive
file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
The green arrows indicate the
elementary test that should
have been successfully
completed BEFORE the test that
the arrow comes from (this is
an indication of the condition,
not the sequence of tests)
• Independent test atom
• Depend on test atom
• The traditional sequence of
tests (moving towards the
arrows of dependence)
Start
Get GPS
position

4. Traditional test flow diagram
BT on BT pair
BT
connect
BT send
small file
BT receive
small file
BT
disconnect
BT
unpair
BT off
Start
• The traditional test does not detect
failures that arise as a result of side
effects of other elements (in this
example, the effect of Wi-Fi on BT).
• Tests like error-free reception of large
files after transferring several small
files, alternating large and small files in
reception and transmission, etc. are
dependent upon the human factor.
• The completeness of the test, it is
necessary to control the timing, that is,
the delays between the start of the
"atomic tests" (some failures, for
example, are detected only when the
elementary tests are executed very
quickly)
BT send
large file
BT receive
large file
Δt2 Δt3
Δt1
Δt…i
Δt…i
Δt…i
Δt…i
Δt…iΔt…iΔtK-1

5. How to represent a complete (exhaustive) set
of test sequences?
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
Get GPS
position
BT send
small file
BT receive
small file
Wi-Fi
disconnect
Wi-Fi
off
• Any test sequence (no matter
how sophisticated it may be) can
be represented as a path in a
graph whose vertices are "test
atoms", and arcs are a sequence
of test execution.
• The weights of arcs connecting
sequentially executed tests will be
the delay time between individual
test acts.
• Multithreading of the device
controlling the execution of the
test based on the multicore make
possible to run several tests
simultaneously (which is not
feasible with the traditional
organization of testing).

6. How to create a complete (exhaustive) set
of test sequences?
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
Get GPS
position
BT send
small file
BT receive
small file
Wi-Fi
disconnect
Wi-Fi
off
Since any arbitrarily complex
test can be represented as a
path in a graph, all possible
tests unambiguously
correspond to a set of paths in
a graph.
That is, a complete set of tests
consisting of no more than “K"
steps can be constructed as
the set of all admissible paths
in a graph whose “n” vertices
are elementary tests, and arcs
are all pairs of tests allowed
for sequential execution.

7. Create an admissible paths
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
Get GPS
position
BT send
small file
BT receive
small file
Wi-Fi
disconnect
Wi-Fi
off
• The valid path must start at
independent "test atom"
• Successful “test atom”, raises a flag
at the vertex, if there is arrow to it
• Vertex with flag can’t be executed
• Following the current peak in the
path may be
• independent peak
• vertex, all launch conditions are
satisfied (all vertices to which
the arrows lead from the target
vertex, have a flag).
• When passing a vertex from which
red or red dotted arrows emanate,
the flags at the vertices to which
these arrows lead are reset.

8. Create an admissible paths example:1
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
BT send
small file
BT receive
small file
Wi-Fi
disconnect
• The first atom test can only be an
independent test. Choose “BT
on”
• If the test is successful, set the
flag in it.
• There are four possibilities for
the next test: two independent
tests (the third independent one
has a flag, which means it cannot
be executed) and two
conditional tests, the condition
of which is the flag just set.
Choose “BT pair”
• If the test is successful, set the
flag in it.

9. Create an admissible paths example:2
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
BT send
small file
BT receive
small file
Wi-Fi
disconnect
• There are five options in the next
step. Choose “BT connect”
• If the test is successful, set the
flag in it.

10. Create an admissible paths example:3
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
BT send
small file
BT receive
small file
Wi-Fi
disconnect
• There are eight options in the
next step. Choose “BT send large
file”
• Since this test is not a condition
of any other, it is not necessary
to set a flag in it.
• The possibilities of choosing the
next “atom test” have not
changed. Choose BT unpair
• Since the current vertex has a
red and a red dashed arrows, we
not only set the flag in it, but
reset the flag in the red arrows

11. Create an admissible paths example:4
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
BT send
small file
BT receive
small file
Wi-Fi
disconnect
• Since the current vertex has a
red and a red dashed arrows, we
not only set the flag in it, but
reset the flags in the red arrows
• Choice options decreased due to
the loss of flags, we choose BT
pair

12. Create an admissible paths example:5
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
BT send
small file
BT receive
small file
Wi-Fi
disconnect
• Set up the connection again and
transfer the file
• And now let's choose for the
next test “Wi-Fi on” and then Wi-
Fi connect

13. Create an admissible paths example:6
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
BT send
small file
BT receive
small file
Wi-Fi
disconnect
• Set up the connection again and
transfer the file
• And now let's choose for the
next test “Wi-Fi on” and then Wi-
Fi connect
• We will send two files of
different sizes and receive the
file by BT, then transfer the file
by Wi-Fi (it’s possible because of
flag in “Wi-Fi connect”)

14. Create an admissible paths example:7
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
BT send
small file
BT receive
small file
Wi-Fi
disconnect
• We will include GPS in the tests
• And transfer several more files in
different ways
• Then disconnect Wi-Fi

15. Create an admissible paths example:8
Wi-Fi
on
BT on
GPS on
BT pair
BT
connect
BT send
large file
BT receive
large file
BT
disconnect
Wi-Fi
connect
Wi-Fi send
file
Wi-Fi
receive file
BT
unpair
BT off
Start
BT send
small file
BT receive
small file
Wi-Fi
disconnect
• We will show the further
construction of the test by
animation
Finish
Δt1

16. Test timeline
BT on BT pair
BT
connect
BT send
large file
Start
BT
unpair
BT pair
BT
connect
BT receive
file
Wi-Fi
on
Wi-Fi
connect
Finish
Δt1 Δt2 Δt3 ……………………………………………………………………………………………….........
• Each arc on the graph on the previous slide (and in any test sequence)
determines not only the order of the tests, but also the time intervals
between the launch of the test atoms.
• Multithreading allows you to ensure that running tests overlap in time and
even run some tests against others

17. Quantitative estimates
• If all atom tests were independent, then the paths would be built on a complete graph, that is, a
graph with (n-1)*n edges.
• In this case, with the length of the test sequence K, at each step there would be a choice of
(n-1) next atom tests, i.e. the number of test sequences would be (n-1) ^ K.
• Each test sequence could be executed M ^ (K-1) in various ways, where M is the number of
valid different intervals between atom tests (the number of different Δti).
• Due to the presence of dependent atom tests and restrictions on the ability to select the next
atom test, the actual number of sequence variants will be significantly less, however, even in
simple cases when the choice of the next atom test will come from a relatively small number
(10..30 options) sets of 10 step sequences at 10 different start intervals will contain about
10 ^ 20 options

18. Test selection
• SOF –stop on fail criteria
• Pass /Fail criteria
• Loop criteria
• Selection definitions:
• “Selection one by one” sequence
• “Random selection” sequence
• Random with priority

19. Conclusions
• A complete set of tests consisting of no more than “K" steps can be
constructed as the set of all admissible paths in a graph whose vertices
are elementary tests, and arcs are all pairs of tests allowed for sequential
execution
• Multi-core multithreading allows you to run multiple tests one after
another with arbitrary delays and even at the same time
• Full test coverage can be built, but cannot be verified due to too many
options. But by prioritizing individual vertices and arcs of the test graph
(based on previously known failures), it is possible to generate sets of
pseudo-random subsets from the full coverage that will detect failures
better than tests planned traditionally

20. Open question

21. The end
Aplit-Soft embedded solutions and services
Yitzhak Tzeel-Krupp
CEO at Aplit-Soft ltd. Company
Moshe Aviv St 6, Or Yehuda, Israel
Email | yatzik.krupp@aplit-soft.com
Mob | +972 54 788 73 75
Site | www.aplit-soft.com