Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The case for consumer-driven contracts
@matthewfellows
Picture the scene...
Monday morning
9am
Phone rings
It’s our VC
We just got the news....
We’re going to build a startup!
Soundify
®
Everyone is doing microservices!
Metric Value
No. teams 1
No. components 1
Test environments 1
Time in pipeline (commit to prod) 5 minutes
Risk 2.5%
Deploy...
Metric Value
No. teams 2
No. components 2
Test environments 4
Time in pipeline (commit to prod) 10 minutes
Risk 5%
Deploym...
Metric Value
No. teams 3
No. components 6
Test environments 18
Time in pipeline (commit to prod) 30 minutes
Risk 10%
Deplo...
ACQUIRED
Metric Value
No. teams 10
No. components 20
Test environments 20 200
Time in pipeline (commit to prod) 120+ minutes
Risk 2...
So what did we learn?
Where did we go wrong?
We pretended microservices were colocated libraries
Cohesive services, loosely coupled
vs
Cohesive teams, tightly coupled by services
Current tooling and strategies are not good enough
“Integration tests are a scam”
- JB Rainsberger
Scam, you say? Justify!
Integrated tests are:
● Slow
● Fragile
● Hard to manage
When they fail, you can’t point to the pro...
Scam, you say? Justify!
“But my integration tests run in Docker,
why can’t I use them?”
- People
Scam, you say? Justify!
“Because Maths”
- Me
Branches per box vs test cases required
2 code branches = 128 tests
5 code branches = 78,125 tests
10 code branches = 10M ...
Good tests have the exact opposite properties
Dictator Driven Contracts
Dictator Driven Contracts
1. Sit in ivory tower and postulate
2. Document perfect API (Swagger, API blueprint etc.)
3. Create said API
4. Publish sa...
Crap, this didn’t work either!
Dictator Consumer Driven Contracts
Benefits?
You’ll know when you break a consumer
You have a form of documentation
You can test things independently
Pact
www.pact.io
Evolved from combining these two principles
Step 1: Define Consumer expectations
Step 1: Define Consumer expectations
Step 1: Define Consumer expectations
Step 1: Define Consumer expectations
Step 2: Verify expectations on Provider
Start with a consumer test
Given “User A exists”
When I Receive “a GET request for user A”
With “these headers and query”
Respond with “200 OK”
And “...
Given “User A does not exist”
When I Receive “a GET request for user A”
Respond with “404 Not Found”
Example
// Setup our expected interactions on the Mock Service.
pact.
AddInteraction().
Given("User billy exists").
UponReceiving(...
// Run the test and verify the interactions.
err := pact.Verify(func() error {
client := Client{
Host: fmt.Sprintf("http:/...
Specification by example
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name fo...
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name fo...
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name fo...
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name fo...
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name fo...
Next publish your pacts
// Publish the Pacts...
p := dsl.Publisher{}
err := p.Publish(types.PublishRequest{
PactURLs: []string{"../pacts/myconsume...
Then verify your provider
// Verify the Provider from tagged Pact files stored in a Pact Broker
response = pact.VerifyProvider(types.VerifyRequest{
...
Verifying a pact between billy and bobby
Given User billy exists
A request to login with user 'billy'
with POST /users/log...
Verifying a pact between billy and bobby
Given User billy exists
A request to login with user 'billy'
with POST /users/log...
Verifying a pact between billy and bobby
Given User billy exists
A request to login with user 'billy'
with POST /users/log...
Verifying a pact between billy and bobby
Given User billy exists
A request to login with user 'billy'
with POST /users/log...
No Silver Bullet
Does not replace communication
What about systems maintained by other teams?
What about systems built in outdated technologies?
Scary outside
world!
3rd Party
Mainframe
Recapping...
● Business impact of integrated tests
● Cause and explanation of those effects
● Alternative approach - isola...
Thank you
- @matthewfellows
Given “The presentation is over”
Upon Receiving “A request for an answer”
With “A valid questi...
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
The case for consumer-driven contracts
Upcoming SlideShare
Loading in …5
×

The case for consumer-driven contracts

443 views

Published on

Taken from the talk given at tconf.io 2016.

It’s 2016 and we still rely on integrated environments and complex E2E test suites to release complex ecosystems also called “software". In this talk, Matt breaks down the arguments for such nonsense and provides a better, faster, safer alternative.

Accompanying notes: http://www.onegeek.com.au/wp-content/uploads/2016/11/tconf-consumer-driven-contracts_summary-notes.pdf

Published in: Technology
  • Be the first to comment

  • Be the first to like this

The case for consumer-driven contracts

  1. 1. The case for consumer-driven contracts @matthewfellows
  2. 2. Picture the scene...
  3. 3. Monday morning
  4. 4. 9am
  5. 5. Phone rings
  6. 6. It’s our VC
  7. 7. We just got the news....
  8. 8. We’re going to build a startup!
  9. 9. Soundify ®
  10. 10. Everyone is doing microservices!
  11. 11. Metric Value No. teams 1 No. components 1 Test environments 1 Time in pipeline (commit to prod) 5 minutes Risk 2.5% Deployments per day 10 CD health check
  12. 12. Metric Value No. teams 2 No. components 2 Test environments 4 Time in pipeline (commit to prod) 10 minutes Risk 5% Deployments per day 10 CD health check
  13. 13. Metric Value No. teams 3 No. components 6 Test environments 18 Time in pipeline (commit to prod) 30 minutes Risk 10% Deployments per day 10 CD health check
  14. 14. ACQUIRED
  15. 15. Metric Value No. teams 10 No. components 20 Test environments 20 200 Time in pipeline (commit to prod) 120+ minutes Risk 20% Deployments per day 1 CD health check
  16. 16. So what did we learn?
  17. 17. Where did we go wrong?
  18. 18. We pretended microservices were colocated libraries
  19. 19. Cohesive services, loosely coupled vs Cohesive teams, tightly coupled by services
  20. 20. Current tooling and strategies are not good enough
  21. 21. “Integration tests are a scam” - JB Rainsberger
  22. 22. Scam, you say? Justify! Integrated tests are: ● Slow ● Fragile ● Hard to manage When they fail, you can’t point to the problem!
  23. 23. Scam, you say? Justify! “But my integration tests run in Docker, why can’t I use them?” - People
  24. 24. Scam, you say? Justify! “Because Maths” - Me
  25. 25. Branches per box vs test cases required 2 code branches = 128 tests 5 code branches = 78,125 tests 10 code branches = 10M tests
  26. 26. Good tests have the exact opposite properties
  27. 27. Dictator Driven Contracts
  28. 28. Dictator Driven Contracts
  29. 29. 1. Sit in ivory tower and postulate 2. Document perfect API (Swagger, API blueprint etc.) 3. Create said API 4. Publish said document to consumers 5. Request dictate consumers update 6. Repeat steps 1-5 How to: Dictator Driven Contracts
  30. 30. Crap, this didn’t work either!
  31. 31. Dictator Consumer Driven Contracts
  32. 32. Benefits?
  33. 33. You’ll know when you break a consumer
  34. 34. You have a form of documentation
  35. 35. You can test things independently
  36. 36. Pact www.pact.io
  37. 37. Evolved from combining these two principles
  38. 38. Step 1: Define Consumer expectations
  39. 39. Step 1: Define Consumer expectations
  40. 40. Step 1: Define Consumer expectations
  41. 41. Step 1: Define Consumer expectations Step 2: Verify expectations on Provider
  42. 42. Start with a consumer test
  43. 43. Given “User A exists” When I Receive “a GET request for user A” With “these headers and query” Respond with “200 OK” And “User A’s details in the body”
  44. 44. Given “User A does not exist” When I Receive “a GET request for user A” Respond with “404 Not Found”
  45. 45. Example
  46. 46. // Setup our expected interactions on the Mock Service. pact. AddInteraction(). Given("User billy exists"). UponReceiving("A request to login with user 'billy'"). WithRequest(dsl.Request{ Method: "POST", Path: "/users/login", Body: loginRequest, }). WillRespondWith(dsl.Response{ Status: 200, Headers: map[string]string{ "Content-Type": "application/json", }, Body: ` { "user": { "name": "billy" } } `, })
  47. 47. // Run the test and verify the interactions. err := pact.Verify(func() error { client := Client{ Host: fmt.Sprintf("http://localhost:%d", pact.Server.Port), } client.loginHandler(rr, req) // Expect User to be set on the Client if client.user == nil { return errors.New("Expected user not to be nil") } return nil }) if err != nil { t.Fatalf("Error on Verify: %v", err) } // Write pact to file `<pwd>/pacts/my_consumer-my_provider.json` // NOTE: This also is a good candidate for use in TestMain(m *testing.M) pact.WritePact()
  48. 48. Specification by example
  49. 49. { "consumer": {"name": "MyConsumer"}, "provider": {"name": "MyProvider"}, "interactions": [ { "description": "Some name for the test", "provider_state": "Some state", "request": { "method": "GET", "path": "/foobar", "headers": { "Content-Type": "application/json" }, "body": { "s": "foo" } }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "body": { "s": "bar" } } } ... ]
  50. 50. { "consumer": {"name": "MyConsumer"}, "provider": {"name": "MyProvider"}, "interactions": [ { "description": "Some name for the test", "provider_state": "Some state", "request": { "method": "GET", "path": "/foobar", "headers": { "Content-Type": "application/json" }, "body": { "s": "foo" } }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "body": { "s": "bar" } } } ... ]
  51. 51. { "consumer": {"name": "MyConsumer"}, "provider": {"name": "MyProvider"}, "interactions": [ { "description": "Some name for the test", "provider_state": "Some state", "request": { "method": "GET", "path": "/foobar", "headers": { "Content-Type": "application/json" }, "body": { "s": "foo" } }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "body": { "s": "bar" } } } ... ]
  52. 52. { "consumer": {"name": "MyConsumer"}, "provider": {"name": "MyProvider"}, "interactions": [ { "description": "Some name for the test", "provider_state": "Some state", "request": { "method": "GET", "path": "/foobar", "headers": { "Content-Type": "application/json" }, "body": { "s": "foo" } }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "body": { "s": "bar" } } } ... ]
  53. 53. { "consumer": {"name": "MyConsumer"}, "provider": {"name": "MyProvider"}, "interactions": [ { "description": "Some name for the test", "provider_state": "Some state", "request": { "method": "GET", "path": "/foobar", "headers": { "Content-Type": "application/json" }, "body": { "s": "foo" } }, "response": { "status": 200, "headers": { "Content-Type": "application/json" }, "body": { "s": "bar" } } } ... ]
  54. 54. Next publish your pacts
  55. 55. // Publish the Pacts... p := dsl.Publisher{} err := p.Publish(types.PublishRequest{ PactURLs: []string{"../pacts/myconsumer-myprovider.json"}, PactBroker: os.Getenv("PACT_BROKER_HOST"), ConsumerVersion: "1.0.0", Tags: []string{"latest", "production"}, BrokerUsername: os.Getenv("PACT_BROKER_USERNAME"), BrokerPassword: os.Getenv("PACT_BROKER_PASSWORD"), })
  56. 56. Then verify your provider
  57. 57. // Verify the Provider from tagged Pact files stored in a Pact Broker response = pact.VerifyProvider(types.VerifyRequest{ ProviderBaseURL: fmt.Sprintf("http://localhost:%d", providerPort), BrokerURL: brokerHost, Tags: []string{"latest", "prod"}, ProviderStatesURL: fmt.Sprintf("http://localhost:%d/states", providerPort), ProviderStatesSetupURL: fmt.Sprintf("http://localhost:%d/setup", providerPort), BrokerUsername: os.Getenv("PACT_BROKER_USERNAME"), BrokerPassword: os.Getenv("PACT_BROKER_PASSWORD"), }) if response.ExitCode != 0 { t.Fatalf("Got %d, Want exit code 0", response.ExitCode) }
  58. 58. Verifying a pact between billy and bobby Given User billy exists A request to login with user 'billy' with POST /users/login returns a response which has status code 200 has a matching body includes headers "Content-Type" with value "application/json" Given User billy does not exist A request to login with user 'billy' with POST /users/login returns a response which has status code 404 includes headers "Content-Type" with value "application/json" ... Finished in 0.03042 seconds 7 examples, 0 failures
  59. 59. Verifying a pact between billy and bobby Given User billy exists A request to login with user 'billy' with POST /users/login returns a response which has status code 200 has a matching body includes headers "Content-Type" with value "application/json" Given User billy does not exist A request to login with user 'billy' with POST /users/login returns a response which has status code 404 includes headers "Content-Type" with value "application/json" ... Finished in 0.03042 seconds 7 examples, 0 failures
  60. 60. Verifying a pact between billy and bobby Given User billy exists A request to login with user 'billy' with POST /users/login returns a response which has status code 200 has a matching body includes headers "Content-Type" with value "application/json" Given User billy does not exist A request to login with user 'billy' with POST /users/login returns a response which has status code 404 includes headers "Content-Type" with value "application/json" ... Finished in 0.03042 seconds 7 examples, 0 failures
  61. 61. Verifying a pact between billy and bobby Given User billy exists A request to login with user 'billy' with POST /users/login returns a response which has status code 200 has a matching body (FAILED - 1) includes headers "Content-Type" with value "application/json" Failures: 1) Verifying a pact between billy and bobby Given User billy exists A request to login with user 'billy' with POST /users/login returns a response which has a matching body Failure/Error: expect(response_body).to match_term expected_response_body, diff_options Actual: {"user":{"user":"billy"}} @@ -1,6 +1,5 @@ { "user": { - "name": "billy" } }
  62. 62. No Silver Bullet
  63. 63. Does not replace communication
  64. 64. What about systems maintained by other teams?
  65. 65. What about systems built in outdated technologies?
  66. 66. Scary outside world! 3rd Party Mainframe
  67. 67. Recapping... ● Business impact of integrated tests ● Cause and explanation of those effects ● Alternative approach - isolation + contracts ● Contract-testing as an approach, Pact as a tool (in this order!)
  68. 68. Thank you - @matthewfellows Given “The presentation is over” Upon Receiving “A request for an answer” With “A valid question” Respond With “A valid answer”

×