Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From Monoliths to Microservices at Realestate.com.au

1,389 views

Published on

Presented by Jon Eaves (REA), Beth Skurrie (Dius) and Evan Bottcher (ThoughtWorks) at Yow! conference December 2014.

Published in: Software
  • Be the first to comment

From Monoliths to Microservices at Realestate.com.au

  1. 1. From monoliths to microservices
  2. 2. @bethesque @evanbottcher @joneaves
  3. 3. Why?
  4. 4. Why? What?
  5. 5. Why? What? How?
  6. 6. Residential Commercial Media & Developer International Group Global Architecture Property The Money People & Experiences
  7. 7. Why?
  8. 8. 1. Long release cycle
  9. 9. 1. Long release cycle 2. Coupled releases
  10. 10. 1. Long release cycle 2. Coupled releases 3. Long defect fix time
  11. 11. Teams at REA are; self-empowered
  12. 12. Teams at REA are; self-empowered value independence
  13. 13. Teams at REA are; self-empowered value independence trusted
  14. 14. Teams at REA are; self-empowered value independence trusted
  15. 15. Vision
  16. 16. rapid deployment
  17. 17. rapid deployment fault tolerant
  18. 18. rapid deployment fault tolerant discoverable
  19. 19. rapid deployment fault tolerant discoverable independent
  20. 20. rapid deployment fault tolerant discoverable independent
  21. 21. Microservices ?
  22. 22. Microservices 1. Narrow responsibility
  23. 23. Microservices 1. Narrow responsibility 2. Small codebase
  24. 24. Microservices 1. Narrow responsibility 2. Small codebase 3. There is no #3
  25. 25. Starting the journey
  26. 26. What?
  27. 27. Residential Commercial Media & Developer International Group
  28. 28. Group
  29. 29. (source: flickr)
  30. 30. (source: flickr)
  31. 31. CRM ERP Customer Relationship Management Enterprise Resource Planning
  32. 32. CRM ERP
  33. 33. Billing Payment CRM ERP
  34. 34. Billing Payment CRM ERP
  35. 35. Billing Contract Sales automation Product rules Pricing Finance
  36. 36. Business Systems
  37. 37. Line of Business Line of Business Line of Business Customers
  38. 38. Line of Business Line of Business Line of Business Customers Ouch!
  39. 39. Billing Contract Sales automation Product rules Pricing Finance
  40. 40. Billing Sales automation Product rules Pricing Finance Contract
  41. 41. Billing Product rules Contract
  42. 42. Billing Product rules Contract New Apps New Apps
  43. 43. Billing Product rules Contract New Apps New Apps New team boundaries?
  44. 44. (source: wikimedia)
  45. 45. Strategic Goals Architectural Principles Design and Delivery Practices Enable scalable business More customers/transactions Self-service for customers Support entry into new markets Flexible operational processes New products and operational processes Support innovation in existing markets Flexible operational processes New products and operational processes Reduce inertia Make choices that favour rapid feedback and change, with reduced dependencies across teams. Eliminate accidental complexity Aggressively retire and replace unnecessarily complex processes, systems, and integrations so that we can focus on the essential complexity. Consistent interfaces and data flows Eliminate duplication of data and create clear systems of record, with consistent integration interfaces. No silver bullets Off the shelf solutions deliver early value but create inertia and complexity. Standard REST/HTTP Encapsulate legacy Eliminate integration databases Consolidate and cleanse data Published integration model Small independent Services Continuous deployment Minimal customisation of COTS/SAAS
  46. 46. Strategic Goals Architectural Principles Design and Delivery Practices Enable scalable business More customers/transactions Self-service for customers Support entry into new markets Flexible operational processes New products and operational processes Support innovation in existing markets Flexible operational processes New products and operational processes Reduce inertia Make choices that favour rapid feedback and change, with reduced dependencies across teams. Eliminate accidental complexity Aggressively retire and replace unnecessarily complex processes, systems, and integrations so that we can focus on the essential complexity. Consistent interfaces and data flows Eliminate duplication of data and create clear systems of record, with consistent integration interfaces. No silver bullets Off the shelf solutions deliver early value but create inertia and complexity. Standard REST/HTTP Encapsulate legacy Eliminate integration databases Consolidate and cleanse data Published integration model Small independent Services Continuous deployment Minimal customisation of COTS/SAAS
  47. 47. Martin’s first law of distributed objects Don’t distribute!
  48. 48. Single service maintainability Fit-for-purpose technology choice Integration Complexity Environment Provisioning Operational Complexity Testing Complexity Microservices Monoliths
  49. 49. How?
  50. 50. Netscaler configs. Firewall rules. Name all the things consistently. Configured package repository. Cron jobs. Service healthcheck Run the specs. Run code hygiene. Keep all the gems up to Configure monitoring. Enable NRPE. Configure apache. Build an RPM. Configure dependencies. Ruby version. Configure Passenger configs. Logrotate. Splunk forwarder. Logger Create users. Configure NewRelic. Load rake tasks. Load Connect to a database. Make connection pooling work. Externalise configuration. Run database migrations. Chkconfigs.
  51. 51. #noops
  52. 52. (source: wikimedia)
  53. 53. (source: flickr.com) #devops
  54. 54. If you build it, you support it
  55. 55. Integration Complexity Environment Provisioning Operational Complexity Testing Complexity Single service maintainability Fit-for-purpose technology choice Team managed infrastructure If you build it, you support it Microservices Monoliths
  56. 56. Time # of Services 3 months
  57. 57. Time # of Services 3 months
  58. 58. Time # of Services 3 months 2 services!
  59. 59. Condor Pricing Contract Proposals Account Manager Assignments
  60. 60. Condor Contract Proposals Pricing Account Manager Assignments
  61. 61. Condor Customer Acceptance Accepted Contracts Adapter Contracts CDM Contract Proposals Pricing Account Manager Assignments
  62. 62. Condor Customer Acceptance Accepted Contracts Adapter Contracts CDM Contract Proposals Pricing Account Manager Assignments
  63. 63. Condor Customer Acceptance Accepted Contracts Adapter Contracts CDM Contract Proposals Pricing Account Manager Assignments Anti-corruption
  64. 64. (source: flickr.om)
  65. 65. ‘Stencil’ (Service Template)
  66. 66. Stencil
  67. 67. Stencil Service Clone
  68. 68. Service Service Stencil Service Service Service Service
  69. 69. Service Service Stencil Service Service Service Service
  70. 70. Service Service Stencil Service Service Service Service Patch
  71. 71. Harvest a service template (but don’t start by writing one)
  72. 72. Success brings its own problems
  73. 73. Test Prod
  74. 74. Prod Test Test Test
  75. 75. Prod Test Test Test
  76. 76. Slow Easy to break Hard to fix Scales BADLY Integration Tests Lots of set up False negatives Lots of infrastructure
  77. 77. Why do we do integration testing?
  78. 78. Why do we do integration testing?
  79. 79. Why do we do integration testing?
  80. 80. Why do we do Bugs integration testing?
  81. 81. Why do we do Bugs Consequences integration testing?
  82. 82. Bugs Consequences Time Don’t
  83. 83. Consequences Time Don’t know
  84. 84. Time Don’t know Monitoring
  85. 85. Time Don’t know Monitoring
  86. 86. Time Don’t know Monitoring Communication
  87. 87. Time Don’t know Monitoring Communication Cause
  88. 88. Time Don’t know Monitoring Communication Cause Logging
  89. 89. Time Don’t know Monitoring Communication Cause Logging Production
  90. 90. Time Don’t know Monitoring Communication Cause Logging Production Approval process
  91. 91. Time Don’t know Monitoring Communication Cause Logging Production Approval process Integration tests
  92. 92. Monitoring Communication Logging Approval process Integration tests
  93. 93. Monitoring Communication Logging Approval process Integration tests ✓ ✓ ✓ ✓
  94. 94. If you build it, you support it
  95. 95. Monitor meaningful things
  96. 96. Just deploy it
  97. 97. Throw away integration tests
  98. 98. The problem with mocks ??? CA BP ??? mock
  99. 99. Test symmetry CA BP mock
  100. 100. Test symmetry CA BP mock
  101. 101. Pact… how it works C P mock
  102. 102. Pact… how it works C P mock
  103. 103. Pact… how it works C P mock pact
  104. 104. Pact… how it works C P mock pact
  105. 105. Pact… how it works C P mock pact
  106. 106. Slow Easy to break Hard to fix Scales BADLY Integration Tests Lots of set up False negatives Lots of infrastructure
  107. 107. Fast Stable Easy to debug Reliable Pact Scales linearly Low setup No extra infrastructure
  108. 108. Arrange - set up mock server
  109. 109. Arrange - set up mock server
  110. 110. Arrange - set up mock server
  111. 111. Arrange - set up expectations
  112. 112. Arrange - set up expectations
  113. 113. Arrange - set up expectations
  114. 114. Act and Assert
  115. 115. Act and Assert
  116. 116. Act and Assert
  117. 117. Act and Assert
  118. 118. Act and Assert
  119. 119. Set up test data
  120. 120. Set up test data Consumer assumed:
  121. 121. Set up test data Consumer assumed: Provider complies:
  122. 122. Set up test data Consumer assumed: Provider complies:
  123. 123. Verify: Fail
  124. 124. Verify: Success!
  125. 125. https://github.com/realestate-com-au/pact @pact_up
  126. 126. Fast feedback in isolation + Early detection + Rapid recovery
  127. 127. Single service maintainability Fit-for-purpose technology choice Team managed infrastructure If you build it, you support it Service Template Consumer Driven Contracts Early detection Rapid Recovery Integration Complexity Environment Provisioning Operational Complexity Testing Complexity Microservices Monoliths
  128. 128. Time # of Services 9 months 15 services
  129. 129. One ‘Poppenservice’ = Two weeks (source: flickr.com)
  130. 130. To the Cloud!
  131. 131. (source: wikimedia)
  132. 132. Tribe
  133. 133. Tribe Squad Squad Delivery Engineering Squad
  134. 134. “We aim to remove hurdles … and ensure the whole team has the knowledge to responsibly operate the system.” - Hydra’s Vision
  135. 135. Single service maintainability Fit-for-purpose technology choice Team managed infrastructure++ If you build it, you support it Service Template Consumer Driven Contracts Early detection (MTTD) Rapid Recovery Integration Complexity Environment Provisioning Operational Complexity Testing Complexity Microservices Monoliths
  136. 136. One ‘Poppenservice’ = Two hours (source: flickr.com)
  137. 137. Time # of Services 2 years 60 services
  138. 138. (source: flickr.com) Work in Progress
  139. 139. Sharing things is hard
  140. 140. Service
  141. 141. Service Service
  142. 142. Service Service Service
  143. 143. Service Service Service Library
  144. 144. Service Service Service Service Service Library
  145. 145. Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Library
  146. 146. Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Service Library
  147. 147. Sharing == Coupling
  148. 148. All of REA My Tribe My Squad
  149. 149. Be cautious about shared stuff (copy/paste may not be wrong)
  150. 150. Service Custodians
  151. 151. Establish Service Custodianship* (*it’s REALLY hard to make work)
  152. 152. Why?
  153. 153. Why? What?
  154. 154. Why? What? How?
  155. 155. Caution.
  156. 156. Microservices is a long term Time # of Services strategy
  157. 157. If you build it, you support it Harvest a service template Early detection of failures Microservices is a long term strategy Fast feedback in isolation Rapid Recovery Delivery Engineering Deliberate sharing Service Custodianship
  158. 158. Thank You! @bethesque @evanbottcher @joneaves

×