Testing Variability-Intensive Systems, tutorial SPLC 2017, part Ii

www.unamur.be
Testing Variability-intensive
Systems (con’t)
SPLC ’17, Sevilla, Spain

2
gilles.perrouin@unamur.be 2
Part I (Until 16h) : sample configurations to test
Feature modelling
Sampling techniques
Discussion on the JHipster case
Part II (16h30-18h) : select behaviours to test
A (short) introduction to behaviour modelling
Behaviour-driven test case selection
Test concretisation
Open Discussion (18h-18h30)
Tutorial Agenda

FTS
model
VIBeS
Abstract
test suite selection
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
[X. Devroey, PhD thesis, 2017]

FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
VIS
Behaviour
Specification
[X. Devroey, PhD thesis, 2017]

Menu
System 
under 
testSpecification
Test case
Output
Pass or fail?

Specification… of behaviour?

87
6
1 3
close
change
tea serveTea
2
pay open
5soda serveSoda

87
6
1 3
close
change
tea serveTea
2
pay open
5soda serveSoda
Insert coin

87
6
1 3
close
change
tea serveTea
2
pay open
5soda serveSoda
Return change

87
6
1 3
close
change
tea serveTea
2
pay open
5soda serveSoda
Choose soda

87
6
1 3
close
change
tea serveTea
2
pay open
5soda serveSoda
Serve soda

87
6
1 3
close
change
tea serveTea
2
pay open
5soda serveSoda
Open compartment

Model Checking
Model
and Property

Model Checking
Model
and Property
Model
Checker

Model Checking
Model
and Property
Model
Checker
+ counterexample

Variability-Intensive Behaviour
Specification
Optional feature
Choose one or both
VendingMachine
v
Tea
t
Free
f
Cancel
c
Soda
s
Beverages
b

Specification
Optional feature
Choose one or both
VendingMachine
v
Tea
t
Free
f
Cancel
c
Soda
s
Beverages
b
• Feature model  
denotes products
• This one has 12

Model Checking

Model Checking
=

Model Checking
==
+ counterexample
+ counterexample

Model Checking
• Too many to model individually
• Too many to check individually
...

Model Checking
• Too many to model individually
• Too many to check individually
...
Lots of products built from the same assets

Specification
soda serveSodafree
(d) Distributing soda for free
skip
pay
soda serveSoda
open
tea serveTea
(b) Selling tea and soda
close
change
pay soda serveSoda open
(a) Basic vending machine
close
change
pay soda
serveSoda open
cancel
return
(c) With a cancel purchase function
close
change
...
Instead of 12 transition systems:  
 
 
 
 
 
 
VendingMachine
v
Tea
t
Free
f
Cancel
c
Soda
s
Beverages
b
[Classen et al., TSE ’13]
[A. Classen, PhD thesis, 2011]
[M. Cordy, PhD thesis, 2014]

Specification
soda serveSodafree
skip
pay
soda serveSoda
open
tea serveTea
close
change
close
change
pay soda
serveSoda open
cancel
return
close
change
...
 
 
 
 
 
 
VendingMachine
v
Tea
t
Free
f
Cancel
c
Soda
s
Beverages
b
87
6
5
1
pay/¬f
3
open/¬f
cancel/creturn/c
close/¬f
change/¬f
free/f
soda/s serveSoda/s
tea/t
skip/f
2
4
serveTea/t
+ companion feature model:
Write 1 featured transition system: 

Specification
soda serveSodafree
skip
pay
soda serveSoda
open
tea serveTea
close
change
close
change
pay soda
serveSoda open
cancel
return
close
change
...
 
 
 
 
 
 
VendingMachine
v
Tea
t
Free
f
Cancel
c
Soda
s
Beverages
b
feature expressions
87
6
5
1
pay/¬f
3
open/¬f
cancel/creturn/c
close/¬f
change/¬f
free/f
soda/s serveSoda/s
tea/t
skip/f
2
4
serveTea/t
+ companion feature model:
Write 1 featured transition system: 

Exploring the state space of an FTS
initial state, reachable
by all products
87
6
5
1 3
cancel / creturn / c
close / v
change / v
free / f soda / s serveSoda / s
tea / t
take / f
2
4
serveTea / t
pay / ¬f open / ¬f
9
take / v

by all products
Transitions:
87
6
5
1 3
close / v
change / v
tea / t
take / f
2
4
serveTea / t
9
take / v
➁ reachable by prods not having f.1
pay/¬f
2

by all products
2
change/v
3 ➂ reachable by prods having v and not having f.
Transitions:
87
6
5
1 3
close / v
change / v
tea / t
take / f
2
4
serveTea / t
9
take / v
pay/¬f
2

by all products
2
change/v
3
soda/s
5 ➄ reachable by prods having v, s and not having f....
Transitions:
87
6
5
1 3
close / v
change / v
tea / t
take / f
2
4
serveTea / t
9
take / v
pay/¬f
2

by all products
2
change/v
1
free/f
3 ➂ reachable by prods having f (or having v and not having f).
3
soda/s
Transitions:
87
6
5
1 3
close / v
change / v
tea / t
take / f
2
4
serveTea / t
9
take / v
pay/¬f
2

by all products
2
change/v
1
free/f
3 ➂ reachable by prods having f (or having v and not having f).
3
soda/s
3
soda/s
5 ➄ reachable by prods having f, s (or having v, s and not having f).
...
Transitions:
87
6
5
1 3
close / v
change / v
tea / t
take / f
2
4
serveTea / t
9
take / v
pay/¬f
2

87
6
5
1
pay/¬f
3
open/¬f
cancel/creturn/c
close/¬f
change/¬f
free/f
soda/s serveSoda/s
tea/t
skip/f
2
4
serveTea/t
=
=
=
=

How much worth is it? (excerpt)
Small SPL, 4 products
0s
8s
17s
25s
33s
42s
50s
P1.1 P1.2 P1.3 P1.4 P1.5 P1.6
Medium SPL, 64
products
0s
25s
50s
75s
100s
125s
150s
P1.1 P1.2 P1.3 P1.4 P1.5 P1.6
x TS 1 FTS (lower is better)

Getting Featured Transition Systems
• Forward engineering:
draw your (UML, Simulink …) models and
automatically derive the FTS
• Reverse engineering:
build back the FTS from previous, logged
executions of your systems

Forward engineering of FTS
case study
C++ code
with IFDEFs
Informal
statecharts
+ variability
out of sync
model
checking
ProVeLines
instance Visual
statecharts
+ variability
Formal -based
DSL for statecharts
+ variability

https://github.com/xdevroey/yami
Reverse engineering of FTS

LOG
Karma_theme
Facebook_plugin
Wpstorecart_plugin
Toolbox_theme
Gtw_plugin
Clockstone_theme
Deep_blue_theme
Max_theme
Avenue_theme
Admin_user
Wp_sed_plugin
Wp_bing_plugin
Hotfix_plugin
Mini_mail_dashboard_widget_plugin
Theme_plugin
Google_calendar_widget_plugin
Zingiri_web_shop_plugin
U_design_theme
Purevision_theme
Akismet_plugin
Mm_forms_community_plugin
Twentyten_theme
OptimizePress_theme
Plugins
Thecartpress_plugin
Sportpress_theme
Braille_plugin
Premium_Gallery_Manager_plugin
Themes
Lazy_seo_plugin
Reflex_gallery_plugin
Maintheme_theme
Tell_a_friend_plugin
Kiddo_theme
a1_flash_gallery_plugin
Yoko_theme
Wp_404_plugin
Formidable_plugin
News_plugin_plugin
Wp_mailinglist_plugin
AgeRRNV_ageRRNV_splot_xml
Likebtn_like_button_plugin
Twentytwelve_theme
Jetpack_plugin
Twentyeleven_theme
Legend:
Optional
Or
FTS

66.155.40.250 - - [08/Nov/2013:11:38:11 +0100]  
"GET /index.php?p=potins&action=aimepas HTTP/1.1"
192.168.0.10 - - [08/Nov/2013:11:38:42 +0100]  
"GET /index.php?p=potins&action=back HTTP/1.1" 
66.155.40.250 - - [08/Nov/2013:11:39:01 +0100]  
"GET /index.php?p=accueil HTTP/1.1"

66.155.40.250 - - [08/Nov/2013:11:38:11 +0100]  
192.168.0.10 - - [08/Nov/2013:11:38:42 +0100]  
66.155.40.250 - - [08/Nov/2013:11:39:01 +0100]  
Session Session

66.155.40.250 - - [08/Nov/2013:11:38:11 +0100]  
192.168.0.10 - - [08/Nov/2013:11:38:42 +0100]  
66.155.40.250 - - [08/Nov/2013:11:39:01 +0100]  
Session Session
2gram
TS

Bigram object used to build the TS
be.unamur.inference.main.Main

Pattern of the log entries in com.google.code.regexp format

Use sessions to train the bigram

be.unamur.inference.main.Main Requests to include

Requests to excludebe.unamur.inference.main.Main

TS
PLUGIN_REGEXP = ".*/wp-content/plugins/([^/]+)/.*";
THEME_REGEXP = ".*/wp-content/themes/([^/]+)/.*";
ADMIN_REGEXP = ".*/wp-admin/.*";

TS
PLUGIN_REGEXP = ".*/wp-content/plugins/([^/]+)/.*";
THEME_REGEXP = ".*/wp-content/themes/([^/]+)/.*";
ADMIN_REGEXP = ".*/wp-admin/.*";
FTS FM

How to define and select test cases for an SPL,  
based on its behaviour?

Model-Based Testing (MBT) process
Test suite
selection
tool
Model
Test
scripts
Mapping
info.
Concretizer
Test execution
environment
System
under
test
adaptor
Abstract
test
suite

Test suite
selection
tool
Model
Test
scripts
Mapping
info.
Concretizer
Test execution
environment
System
under
test
adaptor
Abstract
test
suite
CPTerminal t
Identification id
DirectDebit dd CreditCard cr Online on Offline off PIN pin Signature sig
requires
PaymentSchema ps Connectivity c
[1..*]
opt.
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
Go_online Go_offline
no_go
go_offline / offgo_online / on
Completed update_card_info
update_card_info
remove_card

Test suite
selection
tool
Model
Test
scripts
Mapping
info.
Concretizer
Test execution
environment
System
under
test
adaptor
Abstract
test
suite
CPTerminal t
Identification id
requires
[1..*]
opt.
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
Test caseTest caseTest caseTest caseTest caseTest caseTest caseTest caseTest case
Test case

Test suite
selection
tool
Model
Test
scripts
Mapping
info.
Concretizer
Test execution
environment
System
under
test
adaptor
Abstract
test
suite
CPTerminal t
Identification id
requires
[1..*]
opt.
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
Test case

Test suite
selection
tool
Model
Test
scripts
Mapping
info.
Concretizer
Test execution
environment
System
under
test
adaptor
Abstract
test
suite
CPTerminal t
Identification id
requires
[1..*]
opt.
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
Test case
insert_card

Test suite
selection
tool
Model
Test
scripts
Mapping
info.
Concretizer
Test execution
environment
System
under
test
adaptor
Abstract
test
suite
CPTerminal t
Identification id
requires
[1..*]
opt.
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
TestTestTestTestTestTestTestTestTestTest case
Test case
insert_card

Behavioural MBT of SPLs process
FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
[Devroey et al., ISoLA ‘12]

FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
Structural-driven
[Devroey et al., SPLAT@SPLC ‘14]
Usage-driven
[Devroey et al., VaMoS ’14]
[Devroey et al., SoSyM ’17]
Dissimilarity-driven

FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suiteFeatured Mutants Model (FMM)
[Devroey et al., FSE ‘14]
[Devroey et al., ICSE ‘16]
Equivalent mutants detection
[Devroey et al., ICST ’17]

FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
Jeremy
Vanhecke

FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suite

Feature Model (FM)
CPTerminal t
Identiﬁcation id
requires
[1..*]
opt.

Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_veriﬁed
check_PIN_online
/ on ∧ pin
check_PIN
_ofﬂine
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card

Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.

Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
Featured
Transition
System (FTS)
Feature  
Model
(FM)

Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
Featured
Transition
System (FTS)
Feature  
Model
(FM)
Feature 
expression

(Positive abstract) test case
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_veriﬁed
check_PIN_online
/ on ∧ pin
check_PIN
_ofﬂine
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
t = (insert_card, select_app, check_PIN_online, go_online,
update_card_info, remove_card)

Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
(dd ∨ cr) ∧ (on ∧ pin) ∧ (on) ∧ CNF( )

Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
(dd ∨ cr) ∧ (on ∧ pin) ∧ (on) ∧ CNF( )prod(fts, t) =

Test suite and structural coverage
• Test suite
• s = (t1, t2, …, tn)
• Structural coverage criterion
• Associates an FTS and a test suite to a real
value between 0 and 1

State/all-states
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_veriﬁed
check_PIN_online
/ on ∧ pin
check_PIN
_ofﬂine
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
coverage(fts,s)
=
states visited by s / number of states in fts

Structural coverage criteria
• State/all-states
• Action/all-actions
• Transition/all-transitions
• Transition-pair/all-transition-pairs
• Path/all-paths

Test suite selection
size products
test suite
coverage

size products
test suite
coverage
Minimal test suite

size products
test suite
coverage
P-minimal test suite

Product prioritisation
• P-coverage
• Coverage reached by sp⊆s,
the test cases from s
executable by p
t1
t2
t3
t4
t5
t6
t7
t8
p = prod(fts,{t1,t2,t4,t5})

• P-coverage
• Coverage reached by sp⊆s,
the test cases from s
executable by p
• P-coverage upper
bound
• Product p able to execute 
sp⊆s with the best coverage
t1
t2
t3
t4
t5
t6
t7
t8
p = prod(fts,{t1,t2,t4,t5})
t1
t2
t3
t4
t5
t6
t7
t8
p = prod(fts, {t3,t4,t5,t8})
with best coverage(fts, {t3,t4,t5,t8})

Dissimilarity-driven selection

Dissimilarity between t1 and t2
t1 = (insert_card, select_app, check_PIN_online, go_online, update_card_info, remove_card)
t2 = (insert_card, select_app, check_PIN_offline, go_offline, update_card_info, remove_card)

Dissa

Dissp

Dissa
Dissp

⊗
Dissa
Dissp

Dissimilarity selection
+ k: the number of test cases
+ d: the duration

(1+1) evolutionary algorithm
s initialized with random test cases

Iterates during d

Sorted using local or global distance

New s is built by
replacing last element by
a random test case

If the fitness value is
higher, the new s is kept

• s = (t1, …, tn)
• t1 more dissimilar than t2 (local distance)
• t1 more dissimilar than (t2, …, tn) (global distance)
• P-coverage upper bound
• Product(s) able to execute the longest prefix of s

Init Card_in
Aborted
insert_card/ 1.0
abort/ 0.2
remove_card / 1.0
select_app
/ 0.8
App_init
CH_verified
check_PIN_online
/ 0.6
check_PIN
_offline
/ 0.3
check_
signature
/ 0.1
no_go / 0.1
go_offline / 0.2go_online / 0.7
Completed update_card_info / 1.0
update_card_info / 1.0
remove_card / 1.0

Init Card_in
Aborted
insert_card/ 1.0
abort/ 0.2
remove_card / 1.0
select_app
/ 0.8
App_init
CH_veriﬁed
check_PIN_online
/ 0.6
check_PIN
_ofﬂine
/ 0.3
check_
signature
/ 0.1
no_go / 0.1
remove_card / 1.0
Usage
model

Init Card_in
Aborted
insert_card/ 1.0
abort/ 0.2
remove_card / 1.0
select_app
/ 0.8
App_init
CH_veriﬁed
check_PIN_online
/ 0.6
check_PIN
_ofﬂine
/ 0.3
check_
signature
/ 0.1
no_go / 0.1
remove_card / 1.0
Pr = 0.336

Init Card_in
Aborted
insert_card/ 1.0
abort/ 0.2
remove_card / 1.0
select_app
/ 0.8
App_init
CH_veriﬁed
check_PIN_online
/ 0.6
check_PIN
_ofﬂine
/ 0.3
check_
signature
/ 0.1
no_go / 0.1
remove_card / 1.0
[Devroey et al., VaMoS ’14, SoSyM ’17]

Init Card_in
Aborted
insert_card/ 1.0
abort/ 0.2
remove_card / 1.0
select_app
/ 0.8
App_init
CH_verified
check_PIN_online
/ 0.6
check_PIN
_offline
/ 0.3
check_
signature
/ 0.1
no_go / 0.1
remove_card / 1.0
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
Test case
Selection
Pr[min, max]
[Devroey et al., VaMoS ’14, SoSyM ’17]

Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
Pr = 0.336

CPTerminal t
Identification id
[1..*]
opt.
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
Pr = 0.336

VIBeS
toolbox
DSL
selection
transformation
execution
mutation
core
fexpression
<<uses>>

VIBeS
toolbox
DSL
selection
transformation
execution
mutation
core
fexpression
<<uses>>
Model definition
public class CPTerminalFts extends
FeaturedTransitionSystemDefinition {
@Override
protected void define() {
initial("i");
from("i").action("insert_card")
.to("CardIn");
from("cardIn").action("init_schema")
.fexpr("dd || cr").to("App_init");
…}}
Structural coverage selection
FeaturedTransitionSystem fts =
new CPTerminalFts().getTransitionSystem();
SolverFacade solver =
new Sat4JSolverFacade(dimacsFile);
TestSet set = allStatesSelection(fts, solver);
...
set = from(fts, solver)
.withLocalMaxDistance(
ftsDissimilarity(solver,levenshtein()))
.during(10000) //sec.
.generate(20); //test-cases

FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
insert_card

AbsCon: a QTaste plugin
Jeremy
Vanhecke
https://github.com/modji-be/AbsCon

Assertions and actions
Veriﬁcations and operations (Operations.py)
maps
Interface model inst.
(UiMapping.py)
QTaste data mapping
mechanism
uses
Test API
SUT's interface
CSV (TestData.csv)
<realisation id="Google testing">
…
<tests>
<test>
<action>start</action>
<action>goHomePage</action>
<assert>onHomePage</assert>
<action>inputSearchString</action>
<assert>searchResultsPrinted</assert>
<action>clickLink</action>
<assert>pageLoaded</assert>
<action>exit</action>
</test>
...
</tests>
</realisation>

maps
(UiMapping.py)
QTaste data mapping
mechanism
uses
Test API
SUT's interface
CSV (TestData.csv)
from qtaste import *
from UiMappings import *
#Actions definition
def goHomePage():
googlePage.open()
def inputSearchString():
searchBar.enterValue(testData.getValue("SEARCHVALUE"))
...
#Asserts definition
def searchResultsPrinted():
googlePage.waitForPage()
if (not(navPicture.exists())):
time.sleep(3) # wait for loading and retry
return navPicture.exists()
...
<realisation id="Google testing">
…
<tests>
<test>
<action>start</action>
<action>goHomePage</action>
<assert>onHomePage</assert>
<action>inputSearchString</action>
<assert>searchResultsPrinted</assert>
<action>clickLink</action>
<assert>pageLoaded</assert>
<action>exit</action>
</test>
...
</tests>
</realisation>

maps
(UiMapping.py)
QTaste data mapping
mechanism
uses
Test API
SUT's interface
CSV (TestData.csv)
#Actions definition
def goHomePage():
googlePage.open()
...
#Asserts definition
...

maps
(UiMapping.py)
QTaste data mapping
mechanism
uses
Test API
SUT's interface
CSV (TestData.csv)
#Actions definition
def goHomePage():
googlePage.open()
...
#Asserts definition
...
SEARCHVALUE;LINKTOCLICK;BROWSER;
unamur;UNamur;*firefox;
unamur;UNamur;*chrome;
unamur;UNamur;*opera;
unamur;UNamur;*safari;
ucl;UCL;*firefox;
ucl;UCL;*chrome;
ucl;UCL;*opera;
ucl;UCL;*safari;

maps
(UiMapping.py)
QTaste data mapping
mechanism
uses
Test API
SUT's interface
CSV (TestData.csv)

maps
(UiMapping.py)
QTaste data mapping
mechanism
uses
Test API
SUT's interface
CSV (TestData.csv)
+

maps
(UiMapping.py)
QTaste data mapping
mechanism
uses
Test API
SUT's interface
CSV (TestData.csv)
+ exists(): boolean
+ getValue(): String
+ click()
accessMethod
accessValue
WebPageElement
+ enterValue(String)
WebEditBox
WebButton WebLink
WebRadio
Button
WebPictureWebText
+ start()
+ exit()
WebBrowser
api
WebElement
+ open()
+ close()
+ getTitle(): String
+ waitForPage()
address
timeout
WebPage
*

⊗
Jaccardp Hamminga × avg
Dissp Dissa
Jaccarda Dicea Antidicea Levenshteina
[1..*]
opt.
[1..1]
Product ∧ Action ⇒ ⊗
Dissimilarity Sort
(1+1) EA
Local Global
+ d: the duration
FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
conﬁg. Product
adaptor
derivation
Feature
model
+
Abstract
test suite

⊗
Dissp Dissa
[1..*]
opt.
[1..1]
Dissimilarity Sort
(1+1) EA
Local Global
+ d: the duration
FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
config. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
Init Card_in
Aborted
insert_card/ 1.0
abort/ 0.2
remove_card / 1.0
select_app
/ 0.8
App_init
CH_verified
check_PIN_online
/ 0.6
check_PIN
_offline
/ 0.3
check_
signature
/ 0.1
no_go / 0.1
remove_card / 1.0
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
Test case
Selection

⊗
Dissp Dissa
[1..*]
opt.
[1..1]
Dissimilarity Sort
(1+1) EA
Local Global
+ d: the duration
FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
config. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
Init Card_in
Aborted
insert_card/ 1.0
abort/ 0.2
remove_card / 1.0
select_app
/ 0.8
App_init
CH_verified
check_PIN_online
/ 0.6
check_PIN
_offline
/ 0.3
check_
signature
/ 0.1
no_go / 0.1
remove_card / 1.0
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
Test case
Selection
VIBeS
toolbox
DSL
selection
transformation
execution
mutation
core
fexpression
<<uses>>
Model definition
@Override
initial("i");
.to("CardIn");
…}}
...

⊗
Dissp Dissa
[1..*]
opt.
[1..1]
Dissimilarity Sort
(1+1) EA
Local Global
+ d: the duration
FTS
model
VIBeS
Abstract
Products
prioritization
Mutation analysis
Test
scripts
Mapping
info.
AbsCon
Product
config. Product
adaptor
derivation
Feature
model
+
Abstract
test suite
Init Card_in
Aborted
insert_card/ 1.0
abort/ 0.2
remove_card / 1.0
select_app
/ 0.8
App_init
CH_verified
check_PIN_online
/ 0.6
check_PIN
_offline
/ 0.3
check_
signature
/ 0.1
no_go / 0.1
remove_card / 1.0
Init Card_in
Aborted
insert_card
abort
remove_card
select_app
/ dd ∨ cr
App_init
CH_verified
check_PIN_online
/ on ∧ pin
check_PIN
_offline
/ off ∧ pin
check_
signature
/ ¬dd ∧ sig
no_go
update_card_info
remove_card
CPTerminal t
Identification id
requires
[1..*]
opt.
Test case
Selection
VIBeS
toolbox
DSL
selection
transformation
execution
mutation
core
fexpression
<<uses>>
Model definition
@Override
initial("i");
.to("CardIn");
…}}
...
AbsCon: a QTaste plugin
Jeremy
Vanhecke

VIBeS
https://github.com/xdevroey/vibes

https://amost2018.wordpress.com
14th Workshop on Advances  
in Model Based Testing
 
@ICST 2018, Västerås, Sweden
Paper submission: Jan. 12th, 2018
Workshop date: Apr. 13th, 2018

Testing Variability-Intensive Systems, tutorial SPLC 2017, part Ii

Testing Variability-Intensive Systems, tutorial SPLC 2017, part Ii

Recommended

Recommended

More Related Content

More from XavierDevroey

More from XavierDevroey (7)

Recently uploaded

Recently uploaded (20)

Testing Variability-Intensive Systems, tutorial SPLC 2017, part Ii