AWS re:Invent 2016: From Monolithic to Microservices: Evolving Architecture Patterns in the Cloud (ARC305)

697 views

Published on

Gilt, a global e-commerce company, implemented a sophisticated microservices architecture on AWS to handle millions of customers visiting their site at noon every day. The microservices architecture pattern enables independent service scaling, faster deployments, better fault isolation, and graceful degradation. In this session, Emerson Loureiro, Sr. Software Engineer at Gilt, will share Gilt's experiences and lessons learned during their evolution from a single monolithic Rails application in a traditional data center to more than 300 Scala/Java microservices deployed in the cloud. Derek Chiles, AWS Solutions Architect, will review best practices and recommended architectures for deploying microservices on AWS.

Published in: Technology

AWS re:Invent 2016: From Monolithic to Microservices: Evolving Architecture Patterns in the Cloud (ARC305)

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Emerson Loureiro, Sr. Software Engineer, Gilt Derek Chiles, Tech Leader Team, AWS November 30, 2016 From Monolithic to Microservices Gilt’s Journey To Microservices on AWS ARC305
  2. 2. What to Expect from the Session • Why Microservices? • Migration approaches and considerations • Gilt’s journey to microservices on AWS • Architecture patterns and best practices
  3. 3. Gilt
  4. 4. Gilt
  5. 5. Gilt “How can we scale things up independently?” “How can we arrange teams around strategic initiatives?” “How can we make it easy and fast to push changes to production?”
  6. 6. Gilt Ruby on Rails Jobs memcache Postgres Ruby on Rails Jobs memcache Postgres Product service Order service Cart service … Swift (gilt.com) City Swift (giltcity.com) Taste (gilttaste.com) … Front-endJava services Ruby on Rails Jobs memcache Postgres Product service Java services Web PDP Web Search Web Homepage … gilt.com Scala microservices Order service 201520112007 …
  7. 7. Common Mobile Personali- sation Admin Data Gilt (2) “Department” accounts Existing Data Centre “Legacy VPC” (1) Deploy to VPC Dual 10 Gb direct connect line, 2-ms latency.
  8. 8. Monolithic Microservices • Simple deployments • Inter-module refactoring • Binary failure modes • Vertical scaling • Technology monoculture • Partial deployments • Strong module boundaries • Graceful degradation • Horizontal scaling • Technology diversity
  9. 9. Organization & Ownership
  10. 10. Monolithic Organization Organized on technology capabilities UI Team DBA Team App Logic Team Web Tier App Tier DB Organizational Structure Application Architecture
  11. 11. Microservices Organization Organized around business initiatives Login Registration Order Personalization Accounts team Mobile Personalization team Mobile team
  12. 12. The Teams • Ingredients • Team size: typically 3-5 people
  13. 13. The “A” Team
  14. 14. The Teams • Independent teams who fully own their services: • Requirements • Quality • Deployment • Source code • Technology selection
  15. 15. Architecture
  16. 16. Monolithic Architecture Load Balancer Account Service Cart Service Shipping Service StoreFront UI Browser Database Data Access Service
  17. 17. Account Database Inventory Database Shipping Database Microservices Architecture Load Balancer StoreFront UI Browser Account Service Cart Service Shipping Service Load Balancer Load Balancer Load Balancer
  18. 18. Microservices Architecture Load Balancer StoreFront UI Browser Account Database Account Service Cart Service Inventory Database Shipping Service API Gateway Load Balancer Load Balancer Load Balancer Shipping Database
  19. 19. Auto Scaling Group Typical Service Stack at Gilt ELB DNS Docker registry … Amazon CloudWatch Metrics Amazon SNS Topic EC2 Instance Service “X” VM EC2 Instance Service “X” VM CloudWatch Log Group
  20. 20. Service Instance Types at Gilt
  21. 21. Decomposition
  22. 22. Strangling the Monolith Database StoreFront UI Browser Cart Service Account Service User Service
  23. 23. No New Features on the Monolith Database StoreFront UI Browser New Microservice New DB Cart Service Account Service User Service
  24. 24. Anti-Corruption Layer for Integration ACL Database StoreFront UI Browser New Microservice New DB Cart Service Account Service User Service
  25. 25. Cart Service Account Service Start With a Simple Existing Service… Database StoreFront UI Browser User Microservice User DB New Microservice New DB User Service Client
  26. 26. Account Service User Service Client …Move on to More Complicated Services Cart Microservice Database StoreFront UI Browser User Microservice User DB New Microservice New DB ACL Cart DB Cart Service
  27. 27. …Keep Walking and… Cart Microservice Database StoreFront UI Browser User Microservice User DB New Microservice New DB Account Microservice Cart DB Account DB Account Service User Service Client ACL Cart Service ACL
  28. 28. …The Monolith is Eventually Gone Cart Microservice StoreFront UI Browser User Microservice User DB New Microservice New DB Account Microservice Cart DB Account DB
  29. 29. Data Management
  30. 30. Decentralized Data Stores • Freedom of choice • Low impact schema changes • Independent scalability account- svc cart- svc user- svc Amazon ElastiCache Amazon RDS Amazon DynamoDB RDS
  31. 31. Storage & DB options in AWS Amazon RDS Amazon DynamoDB Amazon ElasticSearch Amazon S3 Amazon Kinesis Amazon ElastiCache Cache NoSQL SQL SearchObject Streaming Amazon Redshift Amazon Glacier
  32. 32. Service
  33. 33. Cross-Service Transactions 1) Avoid them 2) Use an optimistic model • Accept eventual consistency • Design for idempotency • Detect conflicts and resolve later • Ignore conflicts 3) Use a pessimistic model • Create a transaction manager or distributed locking service • Build transaction logic into the services • Rethink your design
  34. 34. Aggregation usr svc Pull model Push model Data Aggregation Application usr svc Data Aggregation Application Pub/Sub Composite Composite Data Service usr account cart account svc cart svc account svc cart svc Pub Sub usr svc account svc cart svc Data Aggregation Application Push Pull
  35. 35. Aggregation usr svc Pull model Push model Data Aggregation Application usr svc Data Aggregation Application Pub/Sub Composite Composite Data Service usr account cart account svc cart svc account svc cart svc Pub Sub usr svc account svc cart svc Data Aggregation Application Push Pull
  36. 36. Aggregation usr svc Pull model Push model Data Aggregation Application usr svc Data Aggregation Application Pub/Sub Composite Composite Data Service usr account cart account svc cart svc account svc cart svc Pub Sub usr svc account svc cart svc Data Aggregation Application Push Pull
  37. 37. Aggregation usr svc Pull model Push model Data Aggregation Application usr svc Data Aggregation Application Pub/Sub Composite Composite Data Service usr account cart account svc cart svc account svc cart svc Pub Sub usr svc account svc cart svc Data Aggregation Application Push Pull
  38. 38. API Discovery & Management
  39. 39. Discovery Option #1 – Convention-based naming shoppingcart.gamma.example.com
  40. 40. Discovery Option #2: Dynamic Discovery
  41. 41. StoreFront UI Account Service Service Registry Cart Service Cart Service 2) Query 3) Request 1) Register Discovery Option #2: Dynamic Discovery
  42. 42. API Management • Enforce API consistency • Monitor key metrics • Implement read-through caching • Provide authentication and authorization • Meter and throttle
  43. 43. Amazon API Gateway • Enforce API consistency • Monitor key metrics • Implement read-through caching • Provide authentication and authorization • Meter and throttle
  44. 44. Amazon API Gateway for Microservices Client API Gateway Monolith on EC2 Microservice 1 (Lambda) Microservice 2 (ELB+EC2) Microservice 3 (AWS Elastic Beanstalk)
  45. 45. DO DON’T • Add new methods • Add optional params • Deprecate • Negotiate disruptive changes • Delete or rename • Add required params • Remove required params • Mandate disruptive changes API Version Management
  46. 46. Deployment
  47. 47. Continuous Delivery & Continuous Deployment Integration & perf tests Build & unit tests beta Prod Monolith Timeline Cart, User Cart, Account Sales, Account, User On the monolith…
  48. 48. Continuous Delivery & Continuous Deployment Create the right build pipeline for each service Integration & perf tests Build & unit tests beta Produser-svc Integration & perf tests Build & unit tests beta gamma Prodcart-svc AWS CodeDeploy Elastic Beanstalk Jenkins TravisCI CircleCIAWS CodePipeline
  49. 49. Deployment Pace cart-svc checkout-svc Timeline Rollback
  50. 50. Phased Rollout
  51. 51. Elastic Beanstalk Application Phased Rollout at Gilt Under the Hood Blue-Green Deployments with Elastic Beanstalk Environment (0.0.1) Instance Instance Instance Environment (0.0.2) Instance Instance Instance ELB
  52. 52. Elastic Beanstalk Application Phased Rollout at Gilt Under the Hood Blue-Green Deployments with Elastic Beanstalk Environment (0.0.1) Instance Instance Instance Environment (0.0.2) Instance Instance Instance ELB
  53. 53. Elastic Beanstalk Application Phased Rollout at Gilt Under the Hood Blue-Green Deployments with Elastic Beanstalk Environment (0.0.1) Instance Instance Instance Environment (0.0.2) Instance Instance Instance ELB
  54. 54. Elastic Beanstalk Application Phased Rollout at Gilt Under the Hood Blue-Green Deployments with Elastic Beanstalk Environment (0.0.1) Instance Instance Instance Environment (0.0.2) Instance Instance Instance ELB
  55. 55. Elastic Beanstalk Application Phased Rollout at Gilt Under the Hood Blue-Green Deployments with Elastic Beanstalk Environment (0.0.2) Instance Instance Instance ELB
  56. 56. Phased Rollout at Gilt Under the Hood CodeDeploy Application Canary Deployment Group Production Deployment GroupDevelopment Deployment Group InstanceInstanceInstanceInstance Production ELBDevelopment ELB Gilt Production Users Production DNSDevelopment DNS Testing in Production with CodeDeploy
  57. 57. Phased Rollout at Gilt Under the Hood CodeDeploy Application Canary Deployment Group Production Deployment GroupDevelopment Deployment Group InstanceInstanceInstanceInstance Production ELBDevelopment ELB Gilt Production Production DNS UsersEngineers Development DNS Testing in Production with CodeDeploy Promote Promote
  58. 58. Phased Rollout at Gilt Under the Hood CodeDeploy Application Canary Deployment Group Production Deployment GroupDevelopment Deployment Group InstanceInstanceInstanceInstance Production ELBDevelopment ELB Gilt Production UsersEngineers & QA Production DNSDevelopment DNS Testing in Production with CodeDeploy Promote Promote Dev, please
  59. 59. Monitoring & Logging
  60. 60. Monitoring • Instance level: Memory usage, GC Cycles • Service level: RPMs, Latency, Error rate • Universe level: Endpoint response times
  61. 61. Monitoring at Gilt
  62. 62. Monitoring at Gilt Remote calls response time Submitted orders CloudWatch
  63. 63. Logging • Push your logs somewhere S3 Elasticsearch CloudWatch Logs
  64. 64. Logging • Push your logs somewhere • Have visibility on the whole life cycle Deployment => instance startup => application => termination S3 Elasticsearch CloudWatch
  65. 65. Logging • Push your logs somewhere • Have visibility on the whole life cycle Deployment => instance startup => application => termination • Separate your logs by service and by instance S3 Elasticsearch CloudWatch
  66. 66. Logging At Gilt Service EC2 Instance Service “X” VM EC2 Instance Service “X” VM CloudWatch Log Group Log Stream Log Stream Errors!
  67. 67. Aggregation Layer Log Aggregation CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream CloudWatch Log Group Log Stream Log Stream + + + + + =
  68. 68. Cleanup Service EC2 Instance Service “X” VM EC2 Instance Service “X” VM CloudWatch Log Group Log Stream Log Stream Service EC2 Instance Service “X” VM EC2 Instance Service “X” VM CloudWatch Log Group Log Stream Log Stream Log Aggregation At Gilt Service EC2 Instance Service “X” VM EC2 Instance Service “X” VM CloudWatch Log Group Log Stream Log Stream LambdaAWS Elasticsearch Kibana Lambda Delete! Index! Timer
  69. 69. Good Citizenship
  70. 70. Service Consumer Tenets • Design for failure • Plan to be throttled • Plan to retry (w/ exponential backoff) • Degrade gracefully • Cache when appropriate
  71. 71. Service Provider Tenets • Publish standard metrics • Publish logs for aggregation • Be prepared to throttle • Implementation details are private • Maintain backwards compatibility
  72. 72. Standardize Logging 2016-09-02T15:03:24+00:00 ui-svc INFO [uuid-123] ... 2016-09-02T15:03:25+00:00 catalog-svc INFO [uuid-123] ... 2016-09-02T15:03:26+00:00 checkout-svc ERROR [uuid-123] ... 2016-09-02T15:03:27+00:00 payment-svc INFO [uuid-123] ... 2016-09-02T15:03:27+00:00 shipping-svc INFO [uuid-123] ... ui-svc catalog-svc checkout- svc shipping- svc payment- svc request correlation id: “uuid-123” correlation id: “uuid-123”
  73. 73. Parallelize Aggregate Service Service 1 Service 2 Service 3 Single request parallel request
  74. 74. Parallelize for { a <- callServiceA() b <- callServiceB() c <- callServiceC(a, b) } yield c val futureA = callServiceA() val futureB = callServiceB() for { a <- futureA b <- futureB c <- callServiceC(a, b) } yield c
  75. 75. Degrade Gracefully checkout-svc Downstream Service 1 Downstream Service 2 Downstream Service 3 Single request Request response Fast response
  76. 76. Cache Locally Remote Service Application Runtime Local Cache
  77. 77. Cache Externally Remote Service Local Service External Cache
  78. 78. Use a Read-Through Cache Local Service Cache Remote Service API Gateway
  79. 79. Private Services Putting It All Together Public Services Web front-end Mobile apps
  80. 80. Recap • Organisation & Ownership • Architecture & Decomposition • Data Management • Discovery • API Management • Deployment • Logging and Monitoring • Good citizenship
  81. 81. Thank you!Thank You!
  82. 82. Remember to complete your evaluations!
  83. 83. Related Sessions

×