Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Testing Microservices

At Findly we know test automation is key for continuous delivery. However, in the context of a microservices architecture, our monolithic end-to-end test suites have still been limiting our ability to achieve a truly "continuous" pace of delivery. This talk will explain the principles, processes and techniques we are now using to build test suites for microservices and enable continuous delivery at Findly.

Presented at Auckland Continuous Delivery meetup, May 2016 (http://www.meetup.com/Auckland-Continuous-Delivery/events/230864194/).

Testing Microservices

  1. 1. Testing Microservices Nathan Jones May 2016 Enabling Continuous Delivery at Findly
  2. 2. About Me Nathan Jones @_ncjones Software Engineer @ Findly Previously Yellow Air New Zealand Propellerhead Orion Health
  3. 3. Summary Multiple test suites per deployable Prefer tests with faster feedback Developers involved in testing
  4. 4. Contents Background Why Testing, Why Microservices, Test Scopes, Test Pyramid, Common Issues Our Approach Types of Test, CD Pipeline, Test Suite Metrics Future Work Mock Services, Partial Envs, Non-Functionals
  5. 5. Background
  6. 6. Continuous Delivery Local CI QA Pre Prod Prod Changeset
  7. 7. Technical Barriers for CD auto deploy high availability monitoring clean code test automationtest automation
  8. 8. Microservices “small, autonomous, services working together” - Sam Newman Building Microservices
  9. 9. Microservices Example Orders Service Payment Service Catalog Service Recommend Service Notification Service
  10. 10. Benefits of Microservices tech choices resilience scalability agility replaceability organizational alignment deployed independentlydeployed independently
  11. 11. Independently Deployed Orders Service Payment Service Catalog Service Recommend Service Notification Service v1.2
  12. 12. Test Scopes Class Unit App Functional System End to end (E2E)
  13. 13. Quantity Scope Unit Functional Test Pyramid E2E
  14. 14. Fast Feedback Faster Unit Functional Later in CD Pipeline E2E
  15. 15. Common Issues
  16. 16. E2E Common Issues Unit “We have both kinds of test”
  17. 17. E2E Common Issues Unit “...and we value end-to-end more”
  18. 18. Orders Service Payment Service Catalog Service Recommend Service Notification Service Monolithic E2E test suite Selenium Test Suite Common Issues
  19. 19. Our Approach
  20. 20. Our Approach Each deployable has multiple test suites: Unit Functional End-to-end
  21. 21. Unit Test What Test functions/objects in isolation from other collaborating components. How Test in-process with mocks + sometimes use real DB (integration test). When Impl: same commit as functional change. Run: every pull request. Who Dev
  22. 22. Test Class Under Test Dependency Mock / Stub Dependency Anatomy of Unit Test System
  23. 23. Functional Test What Test app via its API in isolation from other collaborating components. How Cucumber test suite + real DB + real message broker + mock dependencies. When Spec: first task when story picked up. Impl: before story “done”. Run: every pull request + nightly for real env. Who Spec: PO + Devs + Tester. Impl: Devs (mostly) + Tester.
  24. 24. Test App Under Test HTTP DB Async Message Broker Mock / Stub Dependency (async) Mock / Stub Dependee Service Anatomy of Functional Test System Real Downstream Service
  25. 25. Example Functional Test Spec @MBP-396.AC1 Scenario: Health check Given The application has been started When the health check URL is queried Then it returns a valid status message
  26. 26. Functional Test Guidelines ★ Test suite talks to APIs ★ May use real dependencies ★ Create and clean up data ★ Run on every pull request in containers ★ Run against real HA environment nightly ★ Avoid testing orthogonal permutations ★ Collaboration between product owner, tester and developers
  27. 27. End-to-End Test What Test app in fully integrated environment. How Cucumber tests talking to real neighbouring services. When Impl: before story “done” if new integration. Run: nightly + after each deploy. Who Spec: PO + Devs + Tester. Impl: Devs + Tester (mostly).
  28. 28. Test App Under Test Real Upstream Service Anatomy of E2E Test System Real Downstream Service
  29. 29. Example E2E Test Spec @MBP-1616.AC4 @e2e Scenario: Imported Log10 profile is available from Profile Service Given The log10 e2e org is set up When a profile is posted to the E2E org's Log10 endpoint Then the profile can be retrieved from the Profile Service
  30. 30. CD Pipeline Local CI QA Pre Prod Prod Unit Test + Functional Test (partial) Pull request Manual promote Manual promote Manual promote Unit Test + Functional Test Functional Test E2E Test E2E Test (also run nightly) (also run nightly)
  31. 31. Test Suite Metrics Test Count Run Duration Lines of Code
  32. 32. 0 Test Suite Metrics 125 86 292 109 464 240 E2E Functional Unit Job Scheduler Batch Processor ATS Integrator 1 Test Count 2
  33. 33. n/a Test Suite Metrics 80s 45s 6m10s 50s 3m30s E2E Functional Unit cf. Monolithic E2E (600 selenium tests): 9 hours Run Duration 15s Job Scheduler Batch Processor ATS Integrator 5s5s
  34. 34. Test Suite Metrics 2,704 7,708 5,048 10,753 5,902 E2E Functional Unit Lines of Code Job Scheduler Batch Processor ATS Integrator Prod 12,4127,0262,566 n/a 312 1,056 88
  35. 35. Future Work
  36. 36. Mock HTTP Services Wiremock, MockServer, Martian Consumer Driven Contracts UI Testing: use CDN and mock API server
  37. 37. Partially Provisioned Environments Orders Service Payment Service Catalog Service Recommend Service Notification Service Catalog Service Test Suite
  38. 38. Testing Non-Functionals Perf and Security test automation Recovery from restarts and outages Inject failure: Saboteur, Chaos Monkey
  39. 39. Summary Multiple test suites per deployable Prefer tests with faster feedback Developers involved in testing
  40. 40. Books “How Google Tests Software” James Whittaker “Building Microservices” Sam Newman Further Reading Articles “Testing Strategies in Microservices Architecture” Toby Clemson - Thoughtworks http://martinfowler.com/articles/microservice-testing/ “Just Say No to More End to End Tests” Mike Wacker - Google http://googletesting.blogspot.co.nz/2015/04/just-say- no-to-more-end-to-end-tests.html “Guidelines for Structuring Automated Tests” Sunit Parekh - Thoughtworks https://www.thoughtworks. com/insights/blog/guidelines-structuring-automated- tests

×