Microservice Scars - Alt.net 2hr

642 views

Published on

The slide deck for our recent talk at the alt.Net meetup:

Note: These slides make almost no sense without the presentation, although some have requested the slides, so here they are.

If there are any questions regarding the slides, feel free to contact either Abhaya or Joshua.

Microservice scars:
PageUp is on a journey from monolith to microservices.

This talk is to discuss the lessons we learnt from our first microservice. It has been running in production for 9 months - looking back, we have scars, and we've learnt a lot - lets have a retro!

We will cover all sorts of topics ranging from the technical details of our approach, in terms of technology stack, continuous deployments, to the soft skills - stakeholder management, team dynamic. We talk through our experience, and what we took from it. Something for everyone.

Abhaya Chauhan is a Senior Technical Advisor at PageUp - led the team for PageUp's first microservice.

He is focused on ensuring the company is ready for scale. Reducing time to market and bringing agility back to our product. He loves to focus on delivering pragmatically, and showing value.@AbhayaChauhan

www.abhayachauhan.com

Joshua Toth is a Full Stack Developer at PageUp - A member of the team that produced PageUp's first microservice. He loves learning about new technologies and tackling whatever challenge is presented. He has an interest in security and devops as a culture.@TothJoshuaJ

TothJoshuaJ@gmail.com

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
642
On SlideShare
0
From Embeds
0
Number of Embeds
366
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Microservice Scars - Alt.net 2hr

  1. 1. Microservice Scars: Lessons we learnt Abhaya Chauhan @AbhayaChauhan www.abhayachauhan.com Joshua Toth @TothJoshuaJ
  2. 2. Question Policy
  3. 3. Let’s go on a Journey
  4. 4. Context
  5. 5. PageUp Context
  6. 6. Talent Management SaaS Context
  7. 7. 100% Cloud based AWS + Azure Context
  8. 8. 19 years Context
  9. 9. 25% growth year on year Context
  10. 10. PageUp = Market Leaders Context
  11. 11. The Monolith Context
  12. 12. Fragility The Monolith Context
  13. 13. Agility The Monolith Context
  14. 14. Complexity The Monolith Context
  15. 15. Innovation The Monolith Context
  16. 16. Deployment The Monolith Context
  17. 17. Production Hardened The Monolith Context
  18. 18. #Scale Context
  19. 19. Large Projects Context
  20. 20. Big Rewrite Context
  21. 21. If you can’t build a monolith, what makes you think microservices are the answer? Simon Brown Context
  22. 22. First rule of Distributed Computing Context
  23. 23. Don’t do it. Context
  24. 24. Why Microservices? Context
  25. 25. Reduce time to market Context
  26. 26. Reduce blast radius Context
  27. 27. Enable Evolutionary Architecture Context
  28. 28. Sprint Zero
  29. 29. Decisions Sprint Zero
  30. 30. The stack Sprint Zero
  31. 31. Sprint Zero
  32. 32. Sprint Zero Frontend
  33. 33. Sprint Zero Backend Frontend
  34. 34. Sprint Zero Backend DB Frontend
  35. 35. Sprint Zero Elastic Beanstalk S3
  36. 36. .Net Sprint Zero
  37. 37. Self hosted Sprint Zero
  38. 38. Angular Sprint Zero
  39. 39. DynamoDB Sprint Zero
  40. 40. Pact Sprint Zero
  41. 41. Infancy Sprint Zero
  42. 42. Limitations Sprint Zero
  43. 43. Investment Sprint Zero
  44. 44. Deployment Sprint Zero
  45. 45. Continuous Sprint Zero
  46. 46. Phabricator Sprint Zero
  47. 47. Phabro slides had to be removed. Please contact us for any queries Sprint Zero
  48. 48. Pipeline Sprint Zero
  49. 49. Sprint Zero
  50. 50. Pipeline Sprint Zero
  51. 51. Pipeline Sprint Zero
  52. 52. F# Make (Fake) Sprint Zero
  53. 53. Monolith Scripts Sprint Zero
  54. 54. Sharing != caring Sprint Zero
  55. 55. Clear separation Sprint Zero
  56. 56. Development Practices Sprint Zero
  57. 57. TDD Sprint Zero
  58. 58. BDD Sprint Zero
  59. 59. Sprint Zero
  60. 60. Elastic Beanstalk
  61. 61. Full stack Elastic Beanstalk
  62. 62. Visualization Elastic Beanstalk
  63. 63. Monitoring Elastic Beanstalk
  64. 64. Monitoring Elastic Beanstalk
  65. 65. Monitoring Elastic Beanstalk
  66. 66. Monitoring Elastic Beanstalk
  67. 67. Elastic Beanstalk
  68. 68. Elastic Beanstalk
  69. 69. Elastic Beanstalk
  70. 70. Rebuilds Elastic Beanstalk
  71. 71. No Downtime Elastic Beanstalk
  72. 72. The Hack Elastic Beanstalk
  73. 73. Elastic Beanstalk
  74. 74. Elastic Beanstalk
  75. 75. Elastic Beanstalk
  76. 76. Speed Elastic Beanstalk
  77. 77. Care Elastic Beanstalk
  78. 78. Synchronization Elastic Beanstalk
  79. 79. 50 shades of “fuck we can’t roll this back” Grey Elastic Beanstalk
  80. 80. Scaling Elastic Beanstalk
  81. 81. Authentication
  82. 82. IdentityServer3 (Locksmith) OAuth2
  83. 83. Immediate return OAuth2
  84. 84. Questions
  85. 85. Messaging
  86. 86. Messaging
  87. 87. Messaging
  88. 88. Messaging
  89. 89. Request Response Pain Messaging
  90. 90. Resource intensive Messaging
  91. 91. Fault Prone Messaging
  92. 92. 1 to 1 communication Messaging
  93. 93. Event Driven Architecture Messaging
  94. 94. Webhooks Messaging
  95. 95. Messaging
  96. 96. Messaging
  97. 97. :`( Messaging
  98. 98. Queuing (SQS) Messaging
  99. 99. Resource Intensive Fault Prone 1 - many Communication Messaging
  100. 100. Resource Intensive Fault Prone 1 - many Communication Messaging
  101. 101. Resource Intensive Fault Prone 1 - many Communication Messaging
  102. 102. Resource Intensive Fault Prone 1 - many Communication Messaging
  103. 103. Simple Notification Service (SNS) Messaging
  104. 104. Push messages Messaging
  105. 105. Mobile Devices Messaging
  106. 106. Mobile Devices SMS Messaging
  107. 107. Mobile Devices SMS Emails Messaging
  108. 108. Mobile Devices SMS Emails HTTP Endpoints Messaging
  109. 109. Mobile Devices SMS Emails HTTP Endpoints Lambda Messaging
  110. 110. Mobile Devices SMS Emails HTTP Endpoints Lambda Queues Messaging
  111. 111. Messaging
  112. 112. Messaging
  113. 113. Messaging
  114. 114. Messaging SNS + SQS = Amazing Service Bus
  115. 115. Resource Intensive Fault Prone 1 - many Communication Messaging
  116. 116. Circuit Breaker Messaging
  117. 117. Messaging
  118. 118. CloudWatch Messaging
  119. 119. Historical Events Messaging
  120. 120. A store for events Messaging
  121. 121. DynamoDb
  122. 122. Fully Managed NoSQL DB DynamoDb
  123. 123. Fast and Predictable DynamoDb
  124. 124. Distributed DynamoDb
  125. 125. Key-Value Document DynamoDb
  126. 126. Hash (Range) Partition 1 Partition 2 Partition 3 Partition N Read & Write capacity equally distributed Max 400kb DynamoDb
  127. 127. 1 Read unit = 4KB/s 1 Write unit = 1KB/s DynamoDb
  128. 128. CQRS DynamoDb
  129. 129. DynamoDb
  130. 130. Oh shit! DynamoDb
  131. 131. Write is 20x more expensive DynamoDb
  132. 132. DynamoDb {"Key":"Mauritania","Value":"42803"},{"Key":"Mauritius","Value":"42804"},{"Key":"Mexico","Value":"42805"},{"Key":"Micronesia","Value":"42806"},{"Key":"Monaco"," Value":"42807"},{"Key":"Mongolia","Value":"42808"},{"Key":"Montenegro","Value":"42809"},{"Key":"Montserrat","Value":"42810"},{"Key":"Morocco","Value":"42811"}, {"Key":"Mozambique","Value":"42812"},{"Key":"Namibia","Value":"42813"},{"Key":"Nauru","Value":"42814"},{"Key":"Nepal","Value":"42815"},{"Key":"Netherlands"," Value":"42816"},{"Key":"Netherlands Antilles","Value":"42817"},{"Key":"New Caledonia","Value":"42818"},{"Key":"New Zealand","Value":"42819"},{"Key":"Nicaragua"," Value":"42820"},{"Key":"Niger","Value":"42821"},{"Key":"Nigeria","Value":"42822"},{"Key":"Niue","Value":"42823"},{"Key":"Northern Mariana Islands","Value":"42824"}, {"Key":"Norway","Value":"42825"},{"Key":"Oman","Value":"42826"},{"Key":"Pakistan","Value":"42827"},{"Key":"Palau, Republic Of","Value":"42828"},{"Key":" Palestine","Value":"42829"},{"Key":"Panama","Value":"42830"},{"Key":"Papua New Guinea","Value":"42831"},{"Key":"Paraguay","Value":"42832"},{"Key":"Peru"," Value":"42833"},{"Key":"Philippines","Value":"42834"},{"Key":"Poland","Value":"42835"},{"Key":"Portugal","Value":"42836"},{"Key":"Puerto Rico","Value":"42837"}, {"Key":"Qatar","Value":"42838"},{"Key":"Republic Of Korea","Value":"42839"},{"Key":"Republic Of Moldova","Value":"42840"},{"Key":"Republic Of Yemen","Value":" 42841"},{"Key":"Reunion","Value":"42842"},{"Key":"Romania","Value":"42843"},{"Key":"Russian Federation","Value":"42844"},{"Key":"Rwanda","Value":"42845"}, {"Key":"Saint Kitts And Nevis","Value":"42846"},{"Key":"Saint Lucia","Value":"42847"},{"Key":"Sao Tome & Principe","Value":"42848"},{"Key":"Saudi Arabia","Value":" 42849"},{"Key":"Senegal","Value":"42850"},{"Key":"Serbia","Value":"42851"},{"Key":"Serbia & Monten","Value":"42852"},{"Key":"Seychelles","Value":"42853"},{"Key":" Sierra Leone","Value":"42854"},{"Key":"Singapore","Value":"42855"},{"Key":"Slovakia","Value":"42856"},{"Key":"Slovenia","Value":"42857"},{"Key":"Solomon Islands"," Value":"42858"},{"Key":"Somalia","Value":"42859"},{"Key":"South Africa","Value":"42860"},{"Key":"South Sudan","Value":"42861"},{"Key":"Spain","Value":"42862"}, {"Key":"Sri Lanka","Value":"42863"},{"Key":"St. Vincent And The Grena","Value":"42864"},{"Key":"Sudan","Value":"42865"},{"Key":"Surinam","Value":"42866"},{"Key":" Swaziland","Value":"42867"},{"Key":"Sweden","Value":"42868"},{"Key":"Switzerland","Value":"42869"},{"Key":"Syrian Arab Republic","Value":"42870"},{"Key":" Tajikistan","Value":"42871"},{"Key":"Tanzania, United Rep. Of","Value":"42872"},{"Key":"Tfyr Of Macedonia","Value":"42873"},{"Key":"Thailand","Value":"42874"}, {"Key":"Timor-Leste","Value":"42875"},{"Key":"Togo","Value":"42876"},{"Key":"Tokelau Islands","Value":"42877"},{"Key":"Tonga","Value":"42878"},{"Key":"Trinidad And Tobago","Value":"42879"},{"Key":"Tunisia","Value":"42880"},{"Key":"Turkey","Value":"42881"},{"Key":"Turkmenistan","Value":"42882"},{"Key":"Turks And Caicos Island","Value":"42883"},{"Key":"Mauritania","Value":"42803"},{"Key":"Mauritius","Value":"42804"},{"Key":"Mexico","Value":"42805"},{"Key":"Micronesia","Value":" 42806"},{"Key":"Monaco","Value":"42807"},{"Key":"Mongolia","Value":"42808"},{"Key":"Montenegro","Value":"42809"},{"Key":"Montserrat","Value":"42810"},{"Key":" Morocco","Value":"42811"},{"Key":"Mozambique","Value":"42812"},{"Key":"Namibia","Value":"42813"},{"Key":"Nauru","Value":"42814"},{"Key":"Nepal","Value":" 42815"},{"Key":"Netherlands","Value":"42816"},{"Key":"Netherlands Antilles","Value":"42817"},{"Key":"New Caledonia","Value":"42818"},{"Key":"New Zealand"," Value":"42819"},{"Key":"Nicaragua","Value":"42820"},{"Key":"Niger","Value":"42821"},{"Key":"Nigeria","Value":"42822"},{"Key":"Niue","Value":"42823"},{"Key":" Northern Mariana Islands","Value":"42824"},{"Key":"Norway","Value":"42825"},{"Key":"Oman","Value":"42826"},{"Key":"Pakistan","Value":"42827"},{"Key":"Palau, Republic Of","Value":"42828"},{"Key":"Palestine","Value":"42829"},{"Key":"Panama","Value":"42830"},{"Key":"Papua New Guinea","Value":"42831"},{"Key":" Paraguay","Value":"42832"},{"Key":"Peru","Value":"42833"},{"Key":"Philippines","Value":"42834"},{"Key":"Poland","Value":"42835"},{"Key":"Portugal","Value":"42836"}, {"Key":"Puerto Rico","Value":"42837"},{"Key":"Qatar","Value":"42838"},{"Key":"Republic Of Korea","Value":"42839"},{"Key":"Republic Of Moldova","Value":"42840"}, {"Key":"Republic Of Yemen","Value":"42841"},{"Key":"Reunion","Value":"42842"},{"Key":"Romania","Value":"42843"},{"Key":"Russian Federation","Value":"42844"}, {"Key":"Rwanda","Value":"42845"},{"Key":"Saint Kitts And Nevis","Value":"42846"},{"Key":"Saint Lucia","Value":"42847"},{"Key":"Sao Tome & Principe","Value":" 42848"},{"Key":"Saudi Arabia","Value":"42849"},{"Key":"Senegal","Value":"42850"},{"Key":"Serbia","Value":"42851"},{"Key":"Serbia & Monten","Value":"42852"}, {"Key":"Seychelles","Value":"42853"},{"Key":"Sierra Leone","Value":"42854"},{"Key":"Singapore","Value":"42855"},{"Key":"Slovakia","Value":"42856"},{"Key":" Slovenia","Value":"42857"}
  133. 133. DynamoDb {"Key":"Mauritania","Value":"42803"},{"Key":"Mauritius","Value":"42804"},{"Key":" Mexico","Value":"42805"},{"Key":"Micronesia","Value":"42806"},{"Key":"Monaco"," Value":"42807"},{"Key":"Mongolia","Value":"42808"},{"Key":"Montenegro"," Value":"42809"},{"Key":"Montserrat","Value":"42810"},{"Key":"Morocco","Value":" 42811"},{"Key":"Mozambique","Value":"42812"},{"Key":"Namibia","Value":" 42813"},{"Key":"Nauru","Value":"42814"},{"Key":"Nepal","Value":"42815"},{"Key":" Netherlands","Value":"42816"},{"Key":"Netherlands Antilles","Value":"42817"}, {"Key":"New Caledonia","Value":"42818"},{"Key":"New Zealand","Value":" 42819"},{"Key":"Nicaragua","Value":"42820"},{"Key":"Niger","Value":"42821"}, {"Key":"Nigeria","Value":"42822"},{"Key":"Niue","Value":"42823"},{"Key":"Northern Mariana Islands","Value":"42824"},{"Key":"Norway","Value":"42825"},{"Key":" Oman","Value":"42826"},{"Key":"Pakistan","Value":"42827"},{"Key":"Palau, Republic Of","Value":"42828"},{"Key":"Palestine","Value":"42829"},{"Key":" Panama","Value":"42830"},{"Key":"Papua New Guinea","Value":"42831"},{"Key":" Paraguay","Value":"42832"},{"Key":"Peru","Value":"42833"},{"Key":"Philippines"," Value":"42834"},{"Key":"Poland","Value":"42835"},{"Key":"Portugal","Value":" 42836"},{"Key":"Puerto Rico","Value":"42837"},{"Key":"Qatar","Value":"42838"}, {"Key":"Republic Of Korea","Value":"42839"},{"Key":"Republic Of Moldova"," Value":"42840"},{"Key":"Republic Of Yemen","Value":"42841"},{"Key":"Reunion"," Value":"42842"},{"Key":"Romania","Value":"42843"},{"Key":"Russian Federation","Value":"42844"},{"Key":"Rwanda","Value":"42845"},{"Key":"Saint Kitts And Nevis","Value":"42846"},{"Key":"Saint Lucia","Value":"42847"},{"Key":" Sao Tome & Principe","Value":"42848"},{"Key":"Saudi Arabia","Value":"42849"}, {"Key":"Senegal","Value":"42850"},{"Key":"Serbia","Value":"42851"},{"Key":" Serbia & Monten","Value":"42852"},{"Key":"Seychelles","Value":"42853"},{"Key":" Sierra Leone","Value":"42854"},{"Key":"Singapore","Value":"42855"},{"Key":" Slovakia","Value":"42856"},{"Key":"Slovenia","Value":"42857"},{"Key":"Solomon Islands","Value":"42858"},{"Key":"Somalia","Value":"42859"},{"Key":"South Africa","Value":"42860"},{"Key":"South {"Key":"Papua New Guinea","Value":"42831"},{"Key":"Paraguay","Value":" 42832"},{"Key":"Peru","Value":"42833"},{"Key":"Philippines","Value":"42834"}, {"Key":"Poland","Value":"42835"},{"Key":"Portugal","Value":"42836"},{"Key":" Puerto Rico","Value":"42837"},{"Key":"Qatar","Value":"42838"},{"Key":"Republic Of Korea","Value":"42839"},{"Key":"Republic Of Moldova","Value":"42840"} Static (Read Optimised) Volatile (Write Optimised)
  134. 134. Oh shit #2 DynamoDb
  135. 135. DynamoDb
  136. 136. Recovery Point Objective = 30 minutes DynamoDb
  137. 137. Recovery Time Objective = 1 day DynamoDb
  138. 138. Distributed Triggers DynamoDb
  139. 139. DynamoDb
  140. 140. DynamoDb
  141. 141. DynamoDb
  142. 142. DynamoDb
  143. 143. S3 Versioned Buckets DynamoDb
  144. 144. DynamoDb
  145. 145. DynamoDb
  146. 146. DynamoDb
  147. 147. DynamoDb Version Control DynamoDb
  148. 148. Risk Profile Low High
  149. 149. Team
  150. 150. Skillset
  151. 151. Frontend
  152. 152. First Microservice
  153. 153. Quality
  154. 154. NoSql
  155. 155. Infrastructure
  156. 156. Security / DR
  157. 157. Estimation
  158. 158. Risk Profile at startup
  159. 159. PM 101 -Risk Management
  160. 160. Risk Profile: Team 1. Team availability 2. Experience level of the Team 3. Domain knowledge of the Team 4. Technical skillset of the Team 5. Team flow, size and track record
  161. 161. Risk Profile: Product 1. Product Complexity 2. Product Criticality 3. Product Stability 4. Problem Clarity 5. User perspection / expectations
  162. 162. Risk vs Returns
  163. 163. Risk vs Business Value
  164. 164. Retrospective
  165. 165. Monolith Gateway
  166. 166. Stakeholders
  167. 167. More than you think Stakeholders
  168. 168. Metrics Stakeholders
  169. 169. Transparency Stakeholders
  170. 170. Support Balance
  171. 171. Business value
  172. 172. Team goals
  173. 173. Bake some goods
  174. 174. Time to iterate
  175. 175. Don’t underestimate users
  176. 176. Pair pair pair
  177. 177. Questions
  178. 178. Thank you
  179. 179. https://github.com/PageUpPeopleOrg
  180. 180. https://en.wikipedia.org/wiki/Test-driven_development https://en.wikipedia.org/wiki/Behavior-driven_development https://angularjs.org/ http://fsharp.github.io/FAKE/ http://owin.org/ https://aws.amazon.com/documentation/ https://twitter.com/simonbrown http://phabricator.org/ https://github.com/realestate-com-au/pact https://github.com/IdentityServer/IdentityServer3 http://martinfowler.com/bliki/CQRS.html

×