Hamdy Michael Ayas | Hartmut Fischer | Philipp Leitner | Francisco Gomes de Oliveira Neto Department of Computer Science | Interaction Design & Software Engineering

1. An Empirical Analysis of Microservices Systems
Using Consumer-Driven Contract Testing
Hamdy Michael Ayas | Hartmut Fischer | Philipp Leitner | Francisco Gomes de Oliveira Neto
Department of Computer Science | Interaction Design & Software Engineering
01/09/2022

2. 2022-08-29
What? à Microservices-Based Architecture
2

3. 2022-08-29
What? à Microservices-Based Architecture
3
Structuring systems into:
- loosely coupled pieces
- developed and operated independently
- own individual domains and resources
- communicate to compose a complete
system

4. 2022-08-29
What? à Microservices-Based Architecture
4
Structuring systems into:
- loosely coupled pieces
- developed and operated independently
- own individual domains and resources
- communicate to compose a complete
system

5. 2022-08-29
What? à Microservices-Based Architecture
5
Structuring systems into:
- loosely coupled pieces
- developed and operated independently
- own individual domains and resources
- communicate to compose a complete
system

6. 2022-08-29
What? à Microservices-Based Architecture
6
Structuring systems into:
- loosely coupled pieces
- developed and operated independently
- own individual domains and resources
- communicate to compose a complete
system

7. 2022-08-29
Why? à Microservices based architectures
7
- Focus on one thing and do it well
- Design with multiple points of failure
- Conwell’s law and org. alignment
- Faster releases of functionality
- Independent scaling
- Technological diversity
- Separation of security-critical parts

8. 2022-08-29
How? à Individual development
8
Microservice 1
Microservice 2
Microservice 3
…
Develop
Test
Monitor
…
Develop
Test
Monitor
…
Develop
Test
Monitor
…
Develop
Test
Monitor
…
Develop
Test
Monitor
…
Develop
Test
Monitor
…
time

9. 2022-08-29
How? à Consumer-Driven Contracts
9
Producer Service
Consumer Service
Contract
Contract

10. 2022-08-29
Granularity of Services
10

11. 2022-08-29
Granularity of Services
12

12. 2022-08-29
Granularity of Services
13

13. 2022-08-29
Granularity of Services
14

14. 2022-08-29
Granularity of Services
15
Different structures of services call for different
testing structures
Unit Testing Component Testing
Integration Testing E2E Testing

15. 2022-08-29
Challenges & Gaps
17
New testing
paradigms with
MSAs
How best practices
appear in real-world
implementations
How testing types
come together

16. 2022-08-29
Research objectives
18
Devise testing
architecture of
MSAs
Examine
appearance of best
practices
Investigate
testing strategies
in MSAs

17. 2022-08-29
Research Questions
RQ1: What is the testing architecture of open-source
systems that use MSA and CDC tests?
RQ1.1: What artifacts are used and how do they relate to each other?
RQ1.2: What tools and frameworks implement MSA?
RQ2: Do current implementations of open-source MSAs
follow specific guidelines on MSA testing practices?
19

18. 2022-08-29
Methodology
20

19. 2022-08-29
Methodology
21

20. 2022-08-29
Methodology
22

21. 2022-08-29
Methodology
ID Description Framework # of microservices
P1 IoT application for an intelligent football table Pact 2
P2 Public service platform for online payments Pact 6
P3 Governmental platform for licensing exports & imports Pact 7
P4 MS reference implementation of food delivery service SCC 7
23
22 microservices

22. 2022-08-29
RQ1: Testing
Architecture
24

23. 2022-08-29
Unit Tests
• Scope of one microservice
• Dependent classes replaced
with test double
25

24. 2022-08-29
Integration Tests
• Testing the interactions of an
individual microservice with
external modules
• Containerization is used to test
different transactions
• Not visible in all microservices
26

25. 2022-08-29
Component Test
• The functionality of a single
microservice is tested
• Test by calling different endpoints
27

26. 2022-08-29
System Test
• Mocked actions on UI
• User Journey simulations
28

27. 2022-08-29
RQ1: Testing
Architecture
The testing architecture of the
analysed projects cover all levels
of testing (unit, integration,
component and system-level).
Mocking is used in all levels. Most
projects used component tests as
integration tests, even though they
did not target the interaction
between microservices.
29

28. 2022-08-29
RQ2: Alignment with Guidelines and
Best Practices
Comparison with test pyramid
unit test
component test
integration test*
(contract test)
system
test
flaky
reliable
slow,
expensive
fast,
cheap
number of tests
i
s
o
l
a
t
e
d
i
n
t
e
g
r
a
t
e
d
*isolation depending on implementation
30

29. 2022-08-29
RQ2:
Alignment with
Guidelines and
Best Practices
31

30. 2022-08-29
RQ2:
Alignment with
Guidelines and
Best Practices
32

31. 2022-08-29
RQ2:
Alignment with
Guidelines and
Best Practices
33

32. 2022-08-29
RQ2: Alignment with Guidelines and
Best Practices
unit test
component test
integration test*
(contract test)
system
test
flaky
reliable
slow,
expensive
fast,
cheap
number of tests
i
s
o
l
a
t
e
d
i
n
t
e
g
r
a
t
e
d
*isolation depending on implementation
Open-source microservices systems deviate from the test pyramid and the
proportions suggested in Best Practice. Even though some services align with
the test pyramid, the general picture is that the distinction between integration
and component tests is not so clear.
34

33. 2022-08-29
Implications & Contribution
Call to investigate
further service-level
testing
Adaptation of test
pyramid for systems
using MSA
End-to-End
testing
architecture
35

34. 2022-08-29
Conclusion
36