This document discusses testing strategies for microservices architectures compared to monolithic architectures. Some key points: - Testing microservices is more complex due to distributed and asynchronous nature which requires mocking services and generating test data. It also requires coordinating releases across teams. - During planning and development, testers should understand architecture decisions, service contracts, and help develop automated integration tests. They should identify risks and set expectations. - The definition of done should focus on user outcomes, not just development tasks. It is important for testers to provide input on the definition of done. - Migrating a monolith to microservices gradually while maintaining backwards compatibility and fallbacks is challenging but can be done through careful

1. Break Up the Monolith: Testing
Microservices
Marcus Merrell, Director of Technical Services | Sauce Labs

2. What To Take Away
● Insight into the decisions being made
● Where you fit in
● Actions you can take along the way
● Your responsibility to the org—and your users

3. Microservices
Aren’t Microservices Always Better?
● Pros
○ It was already working
○ Unified tech stack
○ Unified developer/tester
ergonomics
● Cons
○ Harder to fix bugs
○ Hard to make big changes
● Pros
○ Developers in different areas
can use the best tool for them
○ Release independently
● Cons
○ Hard to coordinate multi-system
releases
○ Teams get disconnected
Monolith

4. Decisions, Planning, and Question

5. What is my responsibility during planning?
● Sit in the meetings
● Be a part of the plan
● Understand the decisions
● Educate yourself about the stack
● Establish rapport with everyone
● How is testing different in a microservice?
○ Data modeling/creation
○ Mocking
○ Performance
○ Contracts

6. Is it Testable?

7. Testability
● Can you test it? Are there “hooks” in the code for you?
● Do you have access to the code?
● Can endpoints be added for testability?
● How are the unit tests?
● Basic Authentication in lower environments?
● Are there 3rd party tools/vendors involved?

8. Is the contract published?

11. Are the endpoints versioned?

12. Versioned endpoints pin your
users to a version of the contract...
...allowing you to minimize
unintentional breakage

13. Version

14. Your

15. Endpoints

16. http://yourdomain.io/v2/accounts/create

17. Are you using feature flags?

18. Feature flags allow you to release
to production in an “off” state
...they also give you a
method for rollback

19. Use

20. Feature

21. Flags

23. Are your processes
asynchronous?

24. Understand

25. Synchronicity

26. https://bit.ly/2K94FzF

27. Development

28. What is my responsibility during development?
● Hold team accountable for the previous questions
● Identify risks as they arise
● Set expectations
● Identify in-flows/out-flows of data
● Mock API calls to other microservices

29. What is my responsibility during development?
● Understand how requirements translate to UX
● Test to the contracts
● Understand the unit/integration testing devs are doing
● Understand, assist with or create CI builds
● Ideally, automate tickets as you verify
● Always think about & communicate risks!

30. Normal activities, different in Microservice-land
● Data generation
● Mocking
● Sharing with other teams
● Other teams depend on you and your dates
● Help them understand what can be done in parallel

31. Test Data Generation & Mocking
● You will need to generate data for testing
● You should provide tooling for other teams to do the same with your system
● Integration testing might require Mocking other services
● Mock based on Swagger docs
● Test against dedicated environments (ideally)

32. Deployment/Release

33. Definition of Done
● Give input into the Definition of Done
● Ideally this comes from Product
● If DoD isn’t being written, YOU SHOULD WRITE IT
● Test plans feed directly into the DoD
● DoD defines user/product state, NOT dev state

34. Definition of Done - Examples
“At the end of this milestone, all tickets
will be in Pending Release, test plans
will be executed and passing, and
stakeholders will be alerted as to the
release date. No outstanding bugs at
Sev2 or above.”
“At the end of this milestone, all users
will use the new security procedure for
login and password creation. Legacy
data will still be allowed to flow for older
OS versions until legacy traffic falls
below defined threshold of 0.5%.”

35. Definition of Done - Examples
Bad Example Good Example
“At the end of this milestone, all tickets
will be in Pending Release, test plans
will be executed and passing, and
stakeholders will be alerted as to the
release date. No outstanding bugs at
Sev2 or above.”
“At the end of this milestone, all users
will use the new security procedure for
login and password creation. Legacy
data will still be allowed to flow for older
OS versions until legacy traffic falls
below defined threshold of 0.5%.”

36. Story time!
(but first, some terms)

37. PII - Personally Identifiable Information
https://bit.ly/2KgjUpN

38. JWT - Javascript Web Token
https://jwt.io

39. GraphQL
https://graphql.org/

40. And now, a Monolith
(Not to scale)

41. ...Microserviced

42. In the before time
● We didn’t store PII
● We didn’t need super-strong security
● 8-digit hashed tokens (e.g. user=8cba496e)
● Email Address only
● Pre-GDPR

43. Introducing JWTs!

44. The Initial Plan

45. The “Screenplay”

46. Milestone Sequence – Scene 1

47. Milestone Sequence – Scene 2, DoD

48. Assume Nothing
What did we learn?
● Rollbacks for every step
● Figure out stopping points
● Consider external departments
● Get “head nod” from EVERYONE
● Test until it’s BORING
● Sunset plan for legacy services
● Measure, measure, measure
● Don’t include development timelines
● Have a meaningful retrospective
● Maybe have multiple
Risk Mitigation

49. So how did we do?
● 13 services
● 9 scheduled run-throughs
● 8 teams
● Zero reported defects during week of release

50. More Conclusions
● I lobbied for this to be my full-time job
● I was told, “we should never have to do this again”
● They were right about “should”
● But it kept happening
It will keep happening

51. Conclusions

52. Conclusions
● I spent 30% of this talk applying a band-aid to a monolith?
● Anti-patterns occupy more of your time than “getting it right”
● Microservices solve many problems
● Microservices introduce some problems
● For what it’s worth, I still think it’s a net positive

53. Conclusions
● This is a pendulum, not a gate
● Expect more changes to “conventional wisdom”
● It is possible, even likely, that your org will over-do it
● You have a unique opportunity to guide this conversation

54. Questions?