Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Move fast and consumer driven contract test things

227 views

Published on

At SoundCloud, we’ve found that teams move faster when we’ve moved away from a monolith architecture to one based around microservices. Unfortunately, this new type of architecture has been prone to cascading failures when breaking changes go unnoticed in one of our services’ API’s. These failures have had a devastating impact on our system’s uptime, but we’ve found that we can mitigate some of this risk by introducing consumer driven contract tests.

Consumer driven contract tests allow each consumer service and client to define their expectations and interactions with each provider service upstream, and for provider services to verify all of these contracts as part of their build pipeline. Breakage avoided.

In this talk we’ll go through SoundCloud’s process of breaking the monolith into microservices, then see how PACT-based contract tests were introduced, and discuss some of the challenges of adopting such tests in an already-established microservices culture.

Published in: Software
  • Be the first to comment

Move fast and consumer driven contract test things

  1. 1. Making SoundCloud’s µ-services safe(r) to deploy Move Fast and Consumer-Driven- Contract-Test Things XP Days Ukraine 2017
  2. 2. 2.5 years @ SoundCloud Monetization Team Backend Engineer @alonpeer
  3. 3. About SoundCloud a cloud full of sounds 135M tracks, 175M listeners, 12 hours/minute 300+ services / 50+ teams
  4. 4. Agenda ● Testing, monolith style ● My first steps @ SoundCloud ● Introducing contract tests ● Pactifying our services ● Caveats ● QA Q&A
  5. 5. Testing, monolith style
  6. 6. ● One service, one client ● API changes are easy ● One client team to sync with for migrations ● API deprecation is easy Testing, monolith style The good ol’ days
  7. 7. ● Different requirements per client => Code complexity increases ● Harder to deploy without breaking at least one client Testing, monolith style Mo clients, mo problems
  8. 8. ● More manual QA expensive, slow, prone to human error, doesn’t scale ● More end-to-end tests Maintenance nightmare, flaky, slow, creates bottlenecks Testing, monolith style Mo clients, mo problems
  9. 9. My first steps @ SoundCloud
  10. 10. My first steps @ SoundCloud Whoa, micro-services
  11. 11. My first steps @ SoundCloud Testing strategy E2E Unit Integration
  12. 12. My first steps @ SoundCloud Digging into post-mortems © ofsmallthings
  13. 13. My first steps @ SoundCloud “A lack of trust in the acceptance tests caused us to largely ignore the warnings they generated.” “We couldn’t figure out the impact of the broken acceptance tests, and assumed the only problem was the tests themselves, rather than the underlying code.” “The commit went through as there weren't any tests for the serialisation from the client to the backend.” Digging into post-mortems © ofsmallthings
  14. 14. Introducing contract tests
  15. 15. Introducing contract tests My daily routine © LockeSteerpike
  16. 16. Introducing contract tests My daily routine © rickandmorty.com
  17. 17. Unit tests are not enough Introducing contract tests “#cake” >> { result = call(“cake”) assert(result == “lie”) } GET “/cake”: return “lie”
  18. 18. Unit tests are not enough Introducing contract tests GET “/cake”: return “truth” “#cake” >> { result = call(“cake”) assert(result == “lie”) }
  19. 19. Unit tests are not enough Introducing contract tests import JsonLibFoo GET “/cake”: return toJson(current_state) // { “current_state” : “lie” } import JsonLibFoo “#cake” >> { result = fromJson(call(“cake”)) assert(result == “lie”) }
  20. 20. Unit tests are not enough Introducing contract tests import JsonLibBar GET “/cake”: return toJson(current_state) // { “currentState” : “lie” } import JsonLibBar “#cake” >> { result = fromJson(call(“cake”)) assert(result == “lie”) }
  21. 21. Introducing contract tests How this works? Consumer Provider End-to-End Test
  22. 22. Introducing contract tests How this works? Consumer Provider Unit Test Unit Test
  23. 23. Introducing contract tests How this works? Consumer Provider Unit Test Unit Test
  24. 24. Introducing contract tests How this works? ➢ Requesting “/cake”. ➢ Expecting a JSON response with a key “current_state”. ➢ Asserting deserialization of the response succeeds. import JsonLibFoo GET “/cake”: return toJson(current_state) // { “current_state” : “lie” } Consumer Provider
  25. 25. Pactifying our services http://pact.io
  26. 26. What’s Pact? Pactifying our services ● A family of frameworks designed for consumer driven contract testing. ● Supports JSON over HTTP. ● Impl. for Java, Scala, Go, Ruby, .Net, Swift, JS etc. ● Tests run locally, no external dependencies.
  27. 27. What’s Pact? Pactifying our services Step 1: Define consumer expectations © pact.io
  28. 28. What’s Pact? Pactifying our services Step 2: Verify expectations on provider © pact.io
  29. 29. Consumer: client code (LikesClient)
  30. 30. Consumer: client code (LikesClient)
  31. 31. Consumer: client code (LikesClient)
  32. 32. Consumer: client code (LikesClient)
  33. 33. Consumer: client code (LikesClient)
  34. 34. Consumer: client code (LikesClient)
  35. 35. Consumer: client unit test
  36. 36. Consumer: client unit test
  37. 37. Consumer: client unit test
  38. 38. Consumer: client unit test
  39. 39. Consumer: client contract unit test
  40. 40. Consumer: client contract unit test
  41. 41. Consumer: client contract unit test
  42. 42. Consumer: client contract unit test
  43. 43. Consumer: client contract unit test
  44. 44. Consumer: client contract unit test
  45. 45. Consumer: client contract unit test
  46. 46. Consumer: client contract unit test
  47. 47. Consumer: my-consumer-likes.json
  48. 48. Consumer: my-consumer-likes.json
  49. 49. Consumer: my-consumer-likes.json
  50. 50. Consumer: my-consumer-likes.json
  51. 51. Consumer: my-consumer-likes.json
  52. 52. Consumer: my-consumer-likes.json
  53. 53. Consumer: my-consumer-likes.json
  54. 54. Pactifying our services Provider side verification ● Collect pact files from consumers and verify them all. ● Keep your provider environment isolated. ○ Dockerized databases. ○ Test doubles for upstream services.
  55. 55. Pactifying our services Provider states
  56. 56. Pactifying our services Provider states Likes App Likes DB
  57. 57. Pactifying our services Provider states Set state: “User 1000 has liked 2 tracks” Likes App Likes DB
  58. 58. Pactifying our services Provider states INSERT likes Set state: “User 1000 has liked 2 tracks” Likes App Likes DB
  59. 59. Pactifying our services Provider states INSERT likes Set state: “User 1000 has liked 2 tracks” HTTP Request: GET /likes/1000 Likes App Likes DB
  60. 60. Pactifying our services Provider states HTTP Request: GET /likes/1000 HTTP Response: 200 OK INSERT likes Set state: “User 1000 has liked 2 tracks” Likes App Likes DB
  61. 61. Pactifying our services Provider states Likes App Likes DB HTTP Response: 200 OK INSERT likes Set state: “User 1000 has liked 2 tracks” HTTP Request: GET /likes/2000 HTTP Response: 404 Not Found Set state: “User 2000 has liked no tracks” HTTP Request: GET /likes/1000 DELETE likes
  62. 62. Pactifying our services Pact broker ● Share pact files between projects. ● API for pact frameworks to fetch all pacts by provider. ● Support for versioning and tagging. ● Useful UI and visualization of the network.
  63. 63. Pactifying our services Pact broker © pact broker
  64. 64. Pactifying our services Pact broker © pact broker
  65. 65. Pactifying our services Pact broker © pact broker
  66. 66. Pactifying our services Our CI pipeline Push new change Generate consumer contracts Deploy Upload pacts & tag with ‘prod’ Consumer pipeline
  67. 67. Pactifying our services Our CI pipeline Push new change Deploy Upload pacts & tag with ‘prod’ Consumer pipeline Push new change Verify all ‘prod’-tagged pacts Deploy Provider pipeline Generate consumer contracts
  68. 68. ● Communication is key. Caveats
  69. 69. ● Communication is key. ● New moving parts. Caveats
  70. 70. ● Communication is key. ● New moving parts. ● Learning curve per pact framework. Caveats
  71. 71. ● Communication is key. ● New moving parts. ● Learning curve per pact framework. ● Frameworks are WIP. Caveats
  72. 72. ● Communication is key. ● New moving parts. ● Learning curve per pact framework. ● Frameworks are WIP. ● Provider side setup is time consuming. Caveats
  73. 73. ● Communication is key. ● New moving parts. ● Learning curve per pact framework. ● Frameworks are WIP. ● Provider side setup is time consuming. ● Automating consumer-triggered provider verification. Caveats
  74. 74. ● Communication is key. ● New moving parts. ● Learning curve per pact framework. ● Frameworks are WIP. ● Provider side setup is time consuming. ● Automating consumer-triggered provider verification. ● Adoption is essential. Caveats
  75. 75. Recap
  76. 76. Recap Write contract tests Consum ers
  77. 77. Recap Write contract tests Generate pact files Consum ers
  78. 78. Recap Write contract tests Generate pact files Publish to broker Consum ers
  79. 79. Recap Write contract tests Generate pact files Publish to brokerVerify pacts Consum ers Providers
  80. 80. Recap Write contract tests Generate pact files Publish to brokerVerify pacts Consum ers Providers
  81. 81. QA Q&A @alonpeer

×