Testing & deploying Microservices GeeCon 2014

7,024 views

Published on

What do deployment pipelines look like when your system consists of 10s of different types of services? How do you know what to test before deployment? Should you release a service at a time, or bunch them up? This talk goes into the nitty gritty of managing build,test and release of micro services and also covers the often ignored tradeoff between testing before deployment, and testing afterwards.

Presented at GeeCon 2014

Published in: Technology, Business, Education

Testing & deploying Microservices GeeCon 2014

  1. 1. Testing & Deploying Microservices Sam Newman
  2. 2. @samnewman#geecon 2
  3. 3. @samnewman#geecon 3 http://www.thoughtworks.com/radar/
  4. 4. @samnewman#geecon 4 Sam Newman Building Microservices DESIGNING FINE-GRAINED SYSTEMS
  5. 5. @samnewman#geecon 5
  6. 6. @samnewman#geecon 5 Web Shop Shopping Cart Catalog Registration Customer Finance
  7. 7. @samnewman#geecon 5 Web Shop Shopping Cart Catalog Registration Customer Finance 1000 lines of code or less
  8. 8. @samnewman#geecon 6
  9. 9. @samnewman#geecon 6
  10. 10. @samnewman#geecon 7 V1
  11. 11. @samnewman#geecon 7 V2
  12. 12. @samnewman#geecon 8 Go Java NodeJS Ruby
  13. 13. @samnewman#geecon 8 Go NodeJS Ruby Clojure!
  14. 14. @samnewman#geecon 9
  15. 15. @samnewman#geecon 9
  16. 16. @samnewman#geecon 10 Accounts Returns Invoicing Shipping Inventory Customer Service
  17. 17. @samnewman#geecon 10 Accounts Returns Invoicing Shipping Inventory Customer Service
  18. 18. @samnewman#geecon 11
  19. 19. @samnewman#geecon 11
  20. 20. @samnewman#geecon 11 DB
  21. 21. @samnewman#geecon 12 Unit Service UI MIKE COHN’S TEST PYRAMID
  22. 22. @samnewman#geecon 12 Unit Service UI MIKE COHN’S TEST PYRAMID Increasing Scope More Confidence
  23. 23. @samnewman#geecon 12 Unit Service UI MIKE COHN’S TEST PYRAMID Increasing Scope More Confidence Faster! Better Isolation
  24. 24. @samnewman#geecon 12 Unit Service UI MIKE COHN’S TEST PYRAMID ??? ??? Increasing Scope More Confidence Faster! Better Isolation ???
  25. 25. @samnewman#geecon 13 Small Medium Large GOOGLE’S TEST PYRAMID Increasing Scope More Confidence Faster! Better Isolation
  26. 26. @samnewman#geecon 14 DB Small Large Medium
  27. 27. @samnewman#geecon 14 DB Small Large Medium
  28. 28. @samnewman#geecon 15 DB Small Large Medium
  29. 29. @samnewman#geecon 15 DB Small Large Medium
  30. 30. @samnewman#geecon 16 DB Small Large Medium
  31. 31. @samnewman#geecon 16 DB Small Large Medium
  32. 32. @samnewman#geecon 16 DB Small Large Medium
  33. 33. @samnewman#geecon 1717 Small Medium Large TEST SNOWCONE
  34. 34. @samnewman#geecon 18 Small Medium Large
  35. 35. @samnewman#geecon 19 Small Medium Large Build Tests Tests Source Control … BUILD PIPELINE
  36. 36. @samnewman#geecon 19 Small Medium Large Build Tests Tests Source Control Faster Feedback … BUILD PIPELINE
  37. 37. @samnewman#geecon 19 Small Medium Large Build Tests Tests Source Control Faster Feedback Binary Artifact(s) … BUILD PIPELINE
  38. 38. @samnewman#geecon 19 Small Medium Large Build Tests Tests Source Control Faster Feedback Binary Artifact(s) … BUILD PIPELINE
  39. 39. @samnewman#geecon 20 DB
  40. 40. @samnewman#geecon 20 DB
  41. 41. @samnewman#geecon 21 S/M TestsBuild Large Tests UAT Prod
  42. 42. @samnewman#geecon 21 S/M TestsBuild Large Tests UAT Prod DB Machine CI Node Large Tests Environment Large Tests
  43. 43. @samnewman#geecon 22 S/M TestsBuild Large Tests UAT Prod DB Machine UAT Environment Machine
  44. 44. @samnewman#geecon 23 S/M TestsBuild Large Tests UAT Prod Master DB Machine Production Environment Machine Machine Machine Slave DB
  45. 45. @samnewman#geecon 24 S/M TestsBuild Large Tests UAT Prod More Production Like
  46. 46. @samnewman#geecon 24 S/M TestsBuild Large Tests UAT ProdLarge Tests More Production Like
  47. 47. @samnewman#geecon 24 S/M TestsBuild Large Tests UAT ProdLarge Tests Faster Feedback More Production Like
  48. 48. @samnewman#geecon 25
  49. 49. @samnewman#geecon 26 Customer Service
  50. 50. @samnewman#geecon 26 Customer Service S/M TestsBuild Large Tests
  51. 51. @samnewman#geecon 26 Customer Service S/M TestsBuild Large Tests
  52. 52. @samnewman#geecon 26 Customer Service S/M TestsBuild Large Tests
  53. 53. @samnewman#geecon 27 Customer Service Web Shop Fulfilment Service
  54. 54. @samnewman#geecon 27 Customer Service Web Shop Fulfilment Service Large Medium Small
  55. 55. @samnewman#geecon 28 Customer Service Fulfilment Service Large Medium Small
  56. 56. @samnewman#geecon 28 Customer Service Fulfilment Service Large Medium Small
  57. 57. @samnewman#geecon 28 Customer Service Large Medium Small Fulfilment Service Stub
  58. 58. @samnewman#geecon 29 Mountebank http://www.mbtest.org
  59. 59. @samnewman#geecon 30 mountebank :2525
  60. 60. @samnewman#geecon 30 mountebank :2525
  61. 61. @samnewman#geecon 30 mountebank :2525 :5555
  62. 62. @samnewman#geecon 30 mountebank :2525 :5555 Customer Service
  63. 63. @samnewman#geecon 31 S/M TestsBuild Large Tests
  64. 64. @samnewman#geecon 31 S/M TestsBuild Large Tests Customer Service Fulfilment Service Stub
  65. 65. @samnewman#geecon 32 Customer Service V1 Web Shop Fulfilment Service
  66. 66. @samnewman#geecon 32 Web Shop Customer Service v2 Fulfilment Service
  67. 67. @samnewman#geecon 32 Web Shop Customer Service v2 Fulfilment Service
  68. 68. @samnewman#geecon 32 Web Shop Customer Service v2 Fulfilment Service
  69. 69. @samnewman#geecon 33 S/M TestsBuild Large Tests Customer Service Customer Service v1 Web Shop v1 Production
  70. 70. @samnewman#geecon 33 S/M TestsBuild Large Tests Integration Test Customer Service Customer Service v1 Web Shop v1 Production
  71. 71. @samnewman#geecon 33 S/M TestsBuild Large Tests Integration Test Customer Service Customer Service v1 Web Shop v1 Production Customer Service v2 Web Shop v1 Integration Test
  72. 72. @samnewman#geecon 34 Customer Service v1 Web Shop v1 Production
  73. 73. @samnewman#geecon 34 Customer Service v1 Web Shop v1 Production S/M TestsBuild Large Tests Integration Test Customer Service Customer Service v2
  74. 74. @samnewman#geecon 34 S/M TestsBuild Large TestsWeb Shop Customer Service v1 Web Shop v1 Production S/M TestsBuild Large Tests Integration Test Customer Service Customer Service v2
  75. 75. @samnewman#geecon 34 S/M TestsBuild Large TestsWeb Shop Customer Service v1 Web Shop v1 Production S/M TestsBuild Large Tests Integration Test Customer Service Customer Service v2 Web Shop v2
  76. 76. @samnewman#geecon 34 S/M TestsBuild Large TestsWeb Shop Customer Service v1 Web Shop v1 Production S/M TestsBuild Large Tests Integration Test Customer Service Customer Service v2 Web Shop v2 ???
  77. 77. @samnewman#geecon 35 S/M TestsBuild Large TestsWeb Shop S/M TestsBuild Large Tests Customer Service
  78. 78. @samnewman#geecon 35 S/M TestsBuild Large TestsWeb Shop S/M TestsBuild Large Tests Customer Service Integration Test
  79. 79. @samnewman#geecon 35 S/M TestsBuild Large TestsWeb Shop S/M TestsBuild Large Tests Customer Service Integration Test S/M TestsBuild Large Tests Invoice Service
  80. 80. @samnewman#geecon 35 S/M TestsBuild Large TestsWeb Shop S/M TestsBuild Large Tests Customer Service Integration Test S/M TestsBuild Large Tests Invoice Service S/M TestsBuild Large TestsBasket
  81. 81. @samnewman#geecon 35 S/M TestsBuild Large TestsWeb Shop S/M TestsBuild Large Tests Customer Service Integration Test S/M TestsBuild Large Tests Invoice Service S/M TestsBuild Large TestsBasket S/M TestsBuild Large TestsFulfilment
  82. 82. @samnewman#geecon 36
  83. 83. @samnewman#geecon 36 Browsers
  84. 84. @samnewman#geecon 36 Timing Browsers
  85. 85. @samnewman#geecon 36 Provisioning of Environments Timing Browsers
  86. 86. @samnewman#geecon 36 Provisioning of Environments Networks Timing Browsers
  87. 87. @samnewman#geecon 36 Deployment Provisioning of Environments Networks Timing Browsers
  88. 88. @samnewman#geecon 36 Deployment Provisioning of Environments Networks Timing Browsers Diagnosis
  89. 89. @samnewman#geecon 37
  90. 90. @samnewman#geecon 37 Integration Test
  91. 91. @samnewman#geecon 37 Integration Test Prod…
  92. 92. @samnewman#geecon 37 Integration Test Prod…
  93. 93. @samnewman#geecon 37 Integration Test Prod…
  94. 94. @samnewman#geecon 37 Integration Test Prod…
  95. 95. @samnewman#geecon 38
  96. 96. @samnewman#geecon 38
  97. 97. @samnewman#geecon 39 John Allspaw: “Ops Metametrics” http://slidesha.re/dsSZIr
  98. 98. @samnewman#geecon 39 John Allspaw: “Ops Metametrics” http://slidesha.re/dsSZIr
  99. 99. @samnewman#geecon 40 Integration Test Prod… v1v2 v6 v4 = v10
  100. 100. @samnewman#geecon 40 Integration Test Prod… v1v2 v6 v4 = v10
  101. 101. @samnewman#geecon 41 Danger Will Robinson!
  102. 102. @samnewman#geecon 42 Show Tangle
  103. 103. @samnewman#geecon 43 Golden Rule: Get good at releasing services independently
  104. 104. @samnewman#geecon 44 SO NO INTEGRATION TESTS?
  105. 105. @samnewman#geecon 45
  106. 106. @samnewman#geecon 45
  107. 107. @samnewman#geecon 45 SEMANTIC MONITORING
  108. 108. @samnewman#geecon 46 Customer Service Web Shop
  109. 109. @samnewman#geecon 46 Customer Service Web Shop Small Medium Large
  110. 110. @samnewman#geecon 46 Customer Service Web Shop Small Medium Large
  111. 111. @samnewman#geecon 46 Customer Service Web Shop Small Medium Large Consumer Driven Contracts
  112. 112. @samnewman#geecon 47 Customer Service Web Shop
  113. 113. @samnewman#geecon 47 Customer Service Web Shop Expectations
  114. 114. @samnewman#geecon 47 Customer Service Web Shop Expectations
  115. 115. @samnewman#geecon 47 Customer Service Web Shop Expectations Prod
  116. 116. @samnewman#geecon 47 Customer Service Web Shop Expectations Prod
  117. 117. @samnewman#geecon 48
  118. 118. @samnewman#geecon 48 https://github.com/realestate-com-au/pact
  119. 119. @samnewman#geecon 49 Prod Prod Prod Prod
  120. 120. @samnewman#geecon 49 Prod Prod Prod Prod
  121. 121. @samnewman#geecon 49 Prod Prod Prod Prod QA
  122. 122. @samnewman#geecon 49 Prod Prod Prod Prod QA Good Monitoring
  123. 123. @samnewman#geecon 49 Prod Prod Prod Prod QA Good Monitoring Fast Remediation
  124. 124. @samnewman#geecon 49 Prod Prod Prod Prod QA Good Monitoring Fast Remediation
  125. 125. @samnewman#geecon 50 S/M TestsBuild Large Tests UAT ProdLarge Tests Faster Feedback More Production Like
  126. 126. @samnewman#geecon 50 S/M TestsBuild Large Tests UAT ProdLarge Tests Faster Feedback More Production Like
  127. 127. @samnewman#geecon 50 S/M TestsBuild Large Tests UAT ProdLarge Tests Faster Feedback More Production Like
  128. 128. @samnewman#geecon 51 DB Machine CI Node Large Tests Environment S/M TestsBuild Large Tests UAT ProdLarge TestsLarge Tests
  129. 129. @samnewman#geecon 51 DB Machine CI Node Large Tests Environment DB Machine UAT Environment Machine S/M TestsBuild Large Tests UAT ProdLarge Tests UAT
  130. 130. @samnewman#geecon 51 DB Machine CI Node Large Tests Environment DB Machine UAT Environment Machine Master DB Machine Production Environment Machine Machine Machine Slave DB S/M TestsBuild Large Tests UAT ProdLarge Tests Prod
  131. 131. @samnewman#geecon 52 Faster Feedback More Production Like S/M TestsBuild Large Tests UAT ProdLarge Tests
  132. 132. @samnewman#geecon 52 Faster Feedback More Production Like S/M TestsBuild Large Tests UAT ProdLarge Tests
  133. 133. @samnewman#geecon 53
  134. 134. @samnewman#geecon 54
  135. 135. @samnewman#geecon 55 Ansible Puppet Chef
  136. 136. @samnewman#geecon 55 Ansible Puppet Chef
  137. 137. @samnewman#geecon 55 Ansible Puppet Chef AWS
  138. 138. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean
  139. 139. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean OpenStack
  140. 140. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean OpenStack VMWare
  141. 141. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean OpenStack VMWare Vagrant
  142. 142. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean OpenStack VMWare Vagrant Immutable Servers
  143. 143. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean OpenStack VMWare Vagrant Immutable Servers Fast Spin-up
  144. 144. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean OpenStack VMWare Vagrant Immutable Servers Fast Spin-up Provider Agnostic
  145. 145. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean OpenStack VMWare Vagrant Immutable Servers Fast Spin-up Provider Agnostic Feedback Can Suffer
  146. 146. @samnewman#geecon 55 Ansible Puppet Chef AWS Digital Ocean OpenStack VMWare Vagrant Immutable Servers Fast Spin-up Provider Agnostic Feedback Can Suffer Cycle Time
  147. 147. @samnewman#geecon 56 Prod Prod Prod Prod
  148. 148. @samnewman#geecon 56 Prod Prod Prod Prod Packer Images
  149. 149. @samnewman#geecon 56 Prod Prod Prod Prod “Machine” Service Packer Images
  150. 150. @samnewman#geecon 56 Prod Prod Prod Prod “Machine” Service “Machine” Service Service Packer Images
  151. 151. @samnewman#geecon 56 Prod Prod Prod Prod “Machine” Service “Machine” Service Service Packer Images
  152. 152. @samnewman#geecon 57 S/M TestsBuild Large Tests UAT ProdLarge Tests
  153. 153. @samnewman#geecon 57 S/M TestsBuild Large Tests UAT ProdLarge Tests AWS
  154. 154. @samnewman#geecon 57 S/M TestsBuild Large Tests UAT ProdLarge Tests AWS VMWare
  155. 155. @samnewman#geecon 57 S/M TestsBuild Large Tests UAT ProdLarge Tests AWS VMWare Vagrant
  156. 156. @samnewman#geecon 57 S/M TestsBuild Large Tests UAT ProdLarge Tests AWS VMWare Vagrant
  157. 157. @samnewman#geecon 57 S/M TestsBuild Large Tests UAT ProdLarge Tests AWS VMWare Vagrant
  158. 158. @samnewman#geecon 57 S/M TestsBuild Large Tests UAT ProdLarge Tests AWS VMWare Vagrant
  159. 159. @samnewman#geecon 58 “Machine” Service
  160. 160. @samnewman#geecon 58 “Machine” Service Much Easier To Reason About
  161. 161. @samnewman#geecon 58 “Machine” Service Much Easier To Reason About Easier To Provision (Or Decommission)
  162. 162. @samnewman#geecon 58 “Machine” Service Much Easier To Reason About Easier To Provision (Or Decommission) Fewer Side-effects
  163. 163. @samnewman#geecon 58 “Machine” Service Much Easier To Reason About Easier To Provision (Or Decommission) Fewer Side-effects Cost & Management Overhead!
  164. 164. @samnewman#geecon 58 “Machine” Service Much Easier To Reason About Easier To Provision (Or Decommission) Fewer Side-effects Cost & Management Overhead! AWS Digital Ocean OpenStack
  165. 165. @samnewman#geecon 58 “Machine” Service Much Easier To Reason About Easier To Provision (Or Decommission) Fewer Side-effects Cost & Management Overhead! AWS Digital Ocean OpenStack
  166. 166. @samnewman#geecon 59 STANDARD VIRTUALISATION
  167. 167. @samnewman#geecon 59 Machine STANDARD VIRTUALISATION
  168. 168. @samnewman#geecon 59 Machine Base OS STANDARD VIRTUALISATION
  169. 169. @samnewman#geecon 59 Machine Base OS Hypervisor STANDARD VIRTUALISATION
  170. 170. @samnewman#geecon 59 Machine Base OS Hypervisor VM STANDARD VIRTUALISATION
  171. 171. @samnewman#geecon 59 Machine Base OS Hypervisor VM OS STANDARD VIRTUALISATION
  172. 172. @samnewman#geecon 59 Machine Base OS Hypervisor VM OS Apps STANDARD VIRTUALISATION
  173. 173. @samnewman#geecon 59 Machine Base OS Hypervisor VM OS Apps Packer Image STANDARD VIRTUALISATION
  174. 174. @samnewman#geecon 59 Machine Base OS Hypervisor VM OS Apps VM OS Apps Packer Image STANDARD VIRTUALISATION
  175. 175. @samnewman#geecon 59 Machine Base OS Hypervisor VM OS Apps VM OS Apps VM OS Apps Packer Image STANDARD VIRTUALISATION
  176. 176. @samnewman#geecon 60
  177. 177. @samnewman#geecon 61 CONTAINER VIRTUALISATION
  178. 178. @samnewman#geecon 61 Machine CONTAINER VIRTUALISATION
  179. 179. @samnewman#geecon 61 Machine Base OS CONTAINER VIRTUALISATION
  180. 180. @samnewman#geecon 61 Machine Base OS Container CONTAINER VIRTUALISATION
  181. 181. @samnewman#geecon 61 Machine Base OS Container OS CONTAINER VIRTUALISATION
  182. 182. @samnewman#geecon 61 Machine Base OS Container OS Apps CONTAINER VIRTUALISATION
  183. 183. @samnewman#geecon 61 Machine Base OS Container OS Apps Container OS Apps CONTAINER VIRTUALISATION
  184. 184. @samnewman#geecon 61 Machine Base OS Container OS Apps Container OS Apps Container OS Apps CONTAINER VIRTUALISATION
  185. 185. @samnewman#geecon 61 Machine Base OS Container OS Apps Container OS Apps Container OS Apps CONTAINER VIRTUALISATION Linux Only
  186. 186. @samnewman#geecon 61 Machine Base OS Container OS Apps Container OS Apps Container OS Apps CONTAINER VIRTUALISATION Same Kernel Linux Only
  187. 187. @samnewman#geecon 61 Machine Base OS Container OS Apps Container OS Apps Container OS Apps CONTAINER VIRTUALISATION Same Kernel Linux OnlyFine-grained control
  188. 188. @samnewman#geecon 61 Machine Base OS Container OS Apps Container OS Apps Container OS Apps CONTAINER VIRTUALISATION Same Kernel Linux OnlyFine-grained control Very fast to provision
  189. 189. @samnewman#geecon 62
  190. 190. @samnewman#geecon 63 DOCKER
  191. 191. @samnewman#geecon 63 DOCKER Machine
  192. 192. @samnewman#geecon 63 DOCKER Machine Base OS
  193. 193. @samnewman#geecon 63 DOCKER Machine Base OS Docker
  194. 194. @samnewman#geecon 63 DOCKER Machine Base OS Docker Apps
  195. 195. @samnewman#geecon 63 DOCKER Machine Base OS Docker Apps Apps
  196. 196. @samnewman#geecon 63 DOCKER Machine Base OS Docker Apps Apps Apps
  197. 197. @samnewman#geecon 63 DOCKER Machine Base OS Docker Apps Apps Apps Docker Image Registry
  198. 198. @samnewman#geecon 63 DOCKER Machine Base OS Docker Apps Apps Apps Docker Image Registry
  199. 199. @samnewman#geecon 64
  200. 200. @samnewman#geecon 64
  201. 201. @samnewman#geecon 65 S/M TestsBuild Large Tests UAT ProdLarge Tests
  202. 202. @samnewman#geecon 65 S/M TestsBuild Large Tests UAT ProdLarge Tests Docker Image
  203. 203. @samnewman#geecon 65 S/M TestsBuild Large Tests UAT ProdLarge Tests Docker Image Registry Docker Image
  204. 204. @samnewman#geecon 65 S/M TestsBuild Large Tests UAT ProdLarge Tests Docker Image Registry Docker Image
  205. 205. @samnewman#geecon 65 S/M TestsBuild Large Tests UAT ProdLarge Tests Docker Image Registry Docker Image
  206. 206. @samnewman#geecon 66
  207. 207. @samnewman#geecon 66 Be aware of - and balance - your test Pyramid
  208. 208. @samnewman#geecon 66 Be aware of - and balance - your test Pyramid Understand the balance between testing & rapid remediation
  209. 209. @samnewman#geecon 66 Be aware of - and balance - your test Pyramid Understand the balance between testing & rapid remediation Deploy one thing at a time
  210. 210. @samnewman#geecon 66 Be aware of - and balance - your test Pyramid Understand the balance between testing & rapid remediation Deploy one thing at a time Consider consumer-driven contracts over integration tests
  211. 211. @samnewman#geecon 66 Be aware of - and balance - your test Pyramid Understand the balance between testing & rapid remediation Deploy one thing at a time Consider consumer-driven contracts over integration tests Explore image-based deployments to reduce environment differences
  212. 212. Any questions: @samnewman snewman@thoughtworks.com THANKS!

×