My presentation at EuroSTAR conference in Copenhagen 7-10 June 2022

2. About me
• Holds a Ph.D. degree in Computer Science in 1997 from
Norwegian University of Technology
• 25 years work experience in software testing and QA.
• Founder / CEO / Quality Engineer at QualiTest Norway.
• Currently working as contractor at SpareBank 1 – Master
Data Management team
«Pursuing and seeking for new and innovative way of
doing test and QA in smart and effective manner»
• Test automation; continuous test execution; model-
based test; ML supported test; etc…

3. About the talk
01 Context and motivation
02 MBT implementation
03 Lesson learned

4. MDM at SpareBank 1 (SB1)
SB1-MDM
Public
registers
Updates
Consolidation
SB1
Fagsystemer
SB1
Fagsystemer
Enterprise
systems
MDM-usage:
• 7 millions Customer records
• 25 Consumers and 2 millions requests/day
• 12.000 daily updates from public registers
at real-time
Producers
Exposing
SB1
Systemer
SB1
Systemer
Systems
MDM-API
Consumers
Producers
MDM-solution:
• 12 micro-services exposed to Consumers
• 10 enterprise systems integrated
• 300 rules: data validation, transformation
and merge consolidation
• Heavy batch transactions
«The most updated and best
quality of enterprise data
stored in one single place»

5. Challenges
Extremely high requirement
to data quality
Technical complexity -
many integration interfaces
High frequency of changes
to business rules and domain
models
Increasing number of Consumers –
with new requirements
Increasing number of
Enterprise systems joining
the consolidated platform
Automated regression test suites are
constantly growing and changing
Frequent need to rapidly identify a test
suite for a hot-fix at hand (targeted test)

6. What we want
Improve test effectiveness by smart deriving
of test scope caused by particular changes
Reduce cost of developing and maintaining tests
due to constant requirement changes
Better control of traceability between business
requirements and tests
MBT – Model
based testing ???

7. Model based testing – MBT
Rules /
Requirements
Model
Test-
oracle
Test cases
Executable
tests SUT
Manual
modeling
Automated
generation
Automated
generation
Automated
coupling
Automated
execution
Automated
evaluation
Test tool
Expected benefits:
• Automated (smart) test case
generation
• Traceability between requirements
and tests
• Adjustable test coverage – «right
tests for particular change»
• Cost-efficient maintenance of tests
by changing the models instead of
tests

8. Example
Input parameter:
• 32 input parameters.
• Average 4 possible input values needed to be tested
Business rules: 60
Field-rule:
1. If SSN is set then CustomerType = PER
2. LastName length shall not exceed 256 characters
Cross-field rules:
3. If Citizenship2 != null then Citizenship1 != null
4. If SSN is D-No && PersonStatus = 3 then Sector = 9800
Behaviour-rules:
Changes from system A will override changes from system B
because field´s trust level of B is downgraded after X-days

9. Model based testing – MBT
Rules /
Requirements
Model
Test-
oracle
Test cases
Executable
tests SUT
Manual
modeling
Automated
generation
Automated
generation
Automated
coupling
Automated
execution
Automated
evaluation
Test tool

10. Input parameter
Test case filter
Rule
Equivalence
Class
Boundary
PublicIdentificationNo:
regularSSN
d-No
orgNo
invalidIdentNo
LastName:
length_20
length_255
length_256
length_257
Rule_2:
if (Citizenship1=null && Citizenship2 != null) {
Outcome = false
Error_msg = «Citizenship2 rejected»}
Decision
Table
Logical
Operator
Rule_1:
if (SSN=dNo && PersonStatus=3 && SectorCode = 9800)
Outcome = true
Exclude(IdentNo.d-no ||
SectorCode = ‘9800’ ||
Citizenship1 = ‘NOR’ )
Fixed_value(IdentNo = orgNo &&
MaritalStatus = ’Single’ &&
PersonStatus = ‘Resident’)
Modeling

11. Model based testing – MBT
Rules /
Requirements
Model
Test-
oracle
Test cases
Executable
tests SUT
Manual
modeling
Automated
generation
Automated
generation
Automated
coupling
Automated
execution
Automated
evaluation
Test tool

12. Test case generation
Parameter
=
20
No. possible values ≈ 3
No. of possible permutations = 320 (some combinations are not relevant)
è Still «Test case explosion»
Objective of test case generation:
To derive a set of test cases – when being executed will give a
necessary and sufficient coverage for a particular purpose.
Combinatorial algorithm:
o NWise – 2 | 3 | .. | n
o Cartesian (full coverage)
Model element:
o Parameter
o Rule
o TC-filter
Genererated test cases:
o (o1, a1, b3, c2, ...)
o (o2, a2, b1, c2, ...)
o ...

13. Model based testing – MBT
Rules /
Requirements
Model
Test-
oracle
Test cases
Executable
tests SUT
Manual
modeling
Automated
generation
Automated
generation
Automated
coupling
Automated
execution
Automated
evaluation
Test tool

14. Test case execution
SoapUI test
EcFeed
Test case generation
Algorithms +
Rules +
TC-filters
Expected_output, input_1, input_2, ..., input_n
Expected_output, input_1, input_2, ..., input_n
Expected_output, input_1, input_2, ..., input_n
Expected_output, input_1, input_2, ..., input_n
...
Test parameter binding
Evaluate test result
Export test result
• CSV-fil
• DB tables
• Slack
Execute service API

15. Benefits
Expected benefits:
• Automated (smart) test case generation
• Traceability between requirements and tests
• Adjustable test coverage – «right tests for
particular change»
• Cost-efficient maintenance of tests by changing
the models instead of tests

16. Lesson learned
v Need to have a suitable usecase for MBT implementation.
v Completeness/correctness of the model is crucial – hard to find
errors in models. Defect leakage or false alarm.
MBT technique
v Require good domain knowledge and requirement -> PO task?
v Define sufficient number of rules and filters to facilitate effective test
v Automated tests must be parameterized for easy binding
v Require modeling skill and anticipate overhead
Implementation
v Smooth integration between MBT-tool ecFeed and SoapUI
v Improvement potentiale to make MBT-tool more intuitive and «test-
friendly»
Tool integration

17. Our roadmap
Manual
test
Automated
test
Ad-hoc
Rapid and
time efficient
Adaptive
test
Smart and
learning
Continuous
test
Model based
test
Timely and
event-based
Targeted and
effective
ReadyAPI
+ integration
support

18. Questions?
Email minh.nguyen@qualitest.no
LinkedIn www.linkedin.com/in/minhng67
Web www.qualitest.no