Software development is trending toward building systems using small, autonomous, independently deployable services called microservices. Leveraging microservices makes it easier to add and modify system behavior with minimal or no service interruption. Because they facilitate releasing software early, frequently, and continuously, microservices are especially popular in DevOps. But how do microservices affect software testing and testability? Are there new testing challenges that arise from this paradigm? Or are these simply old challenges disguised as new ones? Join Tariq King as he describes the good, the bad, and the reality of testing microservices. Learn how to develop a microservices testing strategy that fits your organization's needs—and avoids common pitfalls and misunderstandings. Whether you're already using microservices or just considering making the shift, come and engage with Tariq as he brings clarity to testing in a microservices world.

1.
W14
Test
Techniques
5/10/17
15:00
Microservices
Testing
Strategies:
The
Good,
the
Bad,
and
the
Reality
Presented
by:
Tariq
King
Ultimate
Software
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.starwest.techwell.com/

2.
Tariq
King
Tariq
King
is
the
director
of
quality
engineering
at
Ultimate
Software.
With
more
than
fifteen
years
of
experience
in
software
testing
research
and
practice,
Tariq
provides
technical
leadership,
people
leadership,
strategic
direction,
and
staff
training
in
software
quality
and
test
engineering.
He
has
previously
held
positions
of
test
manager,
test
architect,
test
lead,
researcher,
and
college
professor.
Tariq
is
a
frequent
presenter
at
conferences
and
workshops,
has
published
more
than
thirty
research
articles
in
IEEE
and
ACM
sponsored
journals,
and
has
developed
and
taught
software
testing
courses
in
both
industry
and
academia.
Contact
him
via
LinkedIn.

3. MICROSERVICES TESTINGA hero will emerge on 5.10.2017 at the Rosen Center Hotel, Orlando FL.
BADTHE
THE
GOOD
REALITY
AND THE
Tariq King
and
ARCHITECT

4. DEVELOPER
ANALYST

5. TESTER
AGENDA
o INTRODUCTION TO MICROSERVICES
o MICROSERVICES TESTING “REALITY CHECK”
o TESTING STRATEGIES
o UNIT TESTING
o CONTRACT TESTING
o READINESS TESTING
o INTEGRATION TESTING
o SYSTEM TESTING
Show Up
Talk
Leave

6. MICROSERVICES
MICROSERVICES
Functional decomposition of system into small, independently deployable services.
Functional decomposition means vertical slicing
as opposed to horizontal slicing into layers.

7. Functional decomposition of system into small, independently deployable services.
PRESENTATION
APPLICATION LOGIC
DATA ACCESS
MICROSERVICES
Functional decomposition of system into small, independently deployable services.
‘Micro’ refers to small size but there is no
universal definition of small.
As small as possible but as big as
necessary to represent domain.
Manageable by a team of
five to six persons.
MICROSERVICES

8. Functional decomposition of system into small, independently deployable services.
PASSENGER
MANAGEMENT
BILLING NOTIFICATIONS PAYMENTS
TRIP
MANAGEMENT
DRIVER
MANAGEMENT
MICROSERVICES
Functional decomposition of system into small, independently deployable services.
Independently deployable also implies that each
microservice is independently scalable.
MICROSERVICES

9. Functional decomposition of system into small, independently deployable services.
Scale by Using Microservices
Y-Axis (functional decomposition)
Scale by Cloning Services
X-Axis (horizontal duplication)
Scale by Sharding Databases
Z-Axis (data partitioning)
MAXIMUM
SCALABILITY
MONOLITHIC
SYSTEM
MICROSERVICES
Functional decomposition of system into small, independently deployable services.
MICROSERVICES
Single
Responsibility
Encapsulation
(Private)
Monitored
(Observable)
Clustered
(Instanced)
Technology
Agnostic

10. MICROSERVICES TESTING
Makes the dependency relationships
between components more explicit.
Single responsibility allows for more
focused testing and debugging.
Monitoring improves observability.
Clustering supports test set up/debugging.
Isolation and independence facilitate
quicker test set up and faster cycles.
Opportunity to reuse existing knowledge
and create new knowledge and tools.
THE GOOD THE BAD
Natural fear associated with new paradigm.
Additional complexity complicates testing.
(network, messaging, varied technologies)
Inter-service calls over network have higher
cost than in-process calls in monolith.
May lead to an explosion in the number of
tools and technologies being maintained.
If microservice is too fine-grained, overhead
can outweigh its utility (nanoservice).
Over-mocking can lead to unrealistic tests.

11. Isolation
Containment
Explicit Relationships
Knowledge
Extremism
Natural Fear
Overhead
Unanswered Questions
Opportunities
WHAT? WHERE? WHEN?
COMPLEXITY, PERFORMANCE
NANOSERVICE ANTI-PATTERN,
OVER-MOCKING
CHANGE
INTEGRATION, PROCESS
MORE SEAMS
NEW APPROACHES, TOOLS
TEST FOCUS, DEBUGGING REUSE CREATE
THE REALITY
KNOWLEDGE REUSE
TECHNIQUES
METHODS
PRINCIPLES
TOOLS
COMPLEXITY, PERFORMANCE
UNIT TESTING
SERVICE
CONTRACT
SERVICE
READINESS
SERVICE
INTEGRATION
SYSTEM
TESTING PYRAMID

12. UNIT TESTING
1. Know what you are testing.
2. Test results not implementation.
3. Test one thing at a time.
4. Make tests readable and
understandable.
5. Make tests independent
and self-sufficient.
6. Make tests deterministic.
7. Repeat yourself when necessary.
8. Measure code coverage but
focus on test coverage.
GUIDELINES
UNIT TESTINGTUTORIALS
UNIT TESTING:
WHAT EVERY TESTER SHOULD KNOW
BEYOND UNIT TESTING:
MORE OF WHAT TESTERS SHOULD KNOW

13. CONTRACT
MUTUAL AGREEMENT
DEFENSIVE
NO TRUST
Benefit for consumer is an obligation for the
provider, and vice-versa.
Full contract (obligation) for provider is the
sum of all consumer contracts (benefits).
MUTUAL AGREEMENT
CONTRACT
obligation(p) benefit(ci)
n
i=1
=
BENEFITS & OBLIGATIONS

14. TESTING THE PROVIDER
IN ISOLATION
MUTUAL AGREEMENT
CONTRACT Focus on checking that valid
inputs produce correct responses.
No need to test invalid inputs.
=
Garbage In Garbage Out
TESTING THE CONSUMER
IN ISOLATION
MUTUAL AGREEMENT
CONTRACT
Test that each consumer respects its contract.
Use contract to create a stub of the provider.
Invoke consumer behavior that calls the stub.
Consumer Provider Stub
Consumer Contract

15. TESTING CONSUMER-PROVIDER
INTEGRATION
MUTUAL AGREEMENT
CONTRACT
Test each consumer with the actual provider.
Use contract to create a request generator.
Send requests to provider and verify responses.
Consumer Provider
Consumer Contract Request Generator
TEST-RELATED GUIDELINES
MUTUAL AGREEMENT
CONTRACT
o Consumers define their own contracts.
o Provider facilitates resolution to any
contradictions among consumer
contracts.
o As service is updated, provider executes
contract tests from all of its consumers.
o If behavior changes in a way that
violates contract, provider must issue a
new version of the service with a new
contract.

16. DEFENSIVE
NO TRUST
TESTING STRATEGIES
PROVIDER IN ISOLATION
Test using valid and invalid inputs.
CONSUMER-PROVIDER INTEGRATION
Test that consumers handle all the error
responses generated by providers.
Code Impact?Improves Security Reduces Performance
Management Health Logging & Auditing Error Handling
Does service start
up, shut down and
restart gracefully?
Is service up and
running? Are its
dependencies?
Is the service recording
implementation and
domain-level events?
Do the failure retries
or fallbacks behave
as expected?
SERVICE READINESS TESTING

17. SERVICE INTEGRATION TESTING
Intra-Bounded
Context Tests
Inter-Bounded
Context Tests
Test Ordering
Integration
Pairwise
Integration
Neighborhood
Integration
SERVICE INTEGRATION TESTING

18. INTEGRATION TEST ORDERING
INTEGRATION TEST ORDERING
Seeks to find the “optimal” order for integration testing
that reduces the cost of creating stubs.
Assumption:
o Developing stubs is time-consuming and error prone.
Techniques:
o Minimize the number of stubs
o Minimize the overall stub complexity.

19. Order for Minimizing # Stubs?
BILLING
PAYMENTS
NOTIFICATIONS
PASSENGER
MANAGEMENT
DRIVER
MANAGEMENT
TRIP
MANAGEMENT
API
GATEWAY
PASSENGER
WEB UI
DRIVER
WEB UI
Order for Minimizing # Stubs?
BILLING
PAYMENTS
NOTIFICATIONS
PASSENGER
MANAGEMENT
DRIVER
MANAGEMENT
TRIP
MANAGEMENT
API
GATEWAY
PASSENGER
WEB UI
DRIVER
WEB UI
1
2
3
4
5
6
7
8
9

20. Alternative Solution
BILLING
PAYMENTS
NOTIFICATIONS
PASSENGER
MANAGEMENT
DRIVER
MANAGEMENT
TRIP
MANAGEMENT
API
GATEWAY
PASSENGER
WEB UI
DRIVER
WEB UI
3
1
4
2
6
5
8
7
9
PAIRWISE INTEGRATION
o Limit testing to each pair of services.
o Results in one integration test
for each edge in call graph.
o You may eliminate the need
for stubs and drivers by using
the actual code instead.

21. TRIP
MANAGEMENT
PASSENGER
WEB UI
BILLING
PAYMENTS
NOTIFICATIONS
DRIVER
MANAGEMENT
DRIVER
WEB UI
API
GATEWAY PASSENGER
MANAGEMENT
PAIRWISE INTEGRATION
The neighborhood of a
service is the set of services
that are one edge away from
the given service.
That is, all the immediate
predecessor services and
immediate successor
services of a given service.

22. TRIP
MANAGEMENT
PASSENGER
WEB UI
BILLING
PAYMENTS
NOTIFICATIONS
DRIVER
MANAGEMENT
DRIVER
WEB UI
API
GATEWAY PASSENGER
MANAGEMENT
Good at reducing the
number of integration test
sessions required
But…
Fault isolation can be
difficult when the
neighborhood is large.

23. TRIP
MANAGEMENT
PASSENGER
WEB UI
BILLING
PAYMENTS
NOTIFICATIONS
DRIVER
WEB UI
API
GATEWAY PASSENGER
MANAGEMENT
Driver Management’s Neighborhood
DRIVER
MANAGEMENT
?
SYSTEM TESTING
End User Testing
Functional,
User Experience
Performance Testing
Load, Scale
Stable, Reliable
Chaos and Stress
Robust, Secure,
Resilient

24. REST-assured
QUESTIONS?
THANK YOU.
Acknowledgements
Dionny Santiago, Test Architect
Robert Vanderwall, Test Architect
Contact Information
tariq_king@ultimatesoftware.com
www.linkedin.com/in/tariqking