Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Microservices and Amazon ECS

1,610 views

Published on

Deep Dive on Microservices and Amazon ECS

Published in: Business
  • Be the first to comment

Microservices and Amazon ECS

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Johan Broman, Solutions Architect AWS Jude D´Souza, Architect Wrapp 2016-05-04 Deep Dive on Microservices and Amazon ECS
  2. 2. What to Expect from this Session Microservices: What are They? Challenges of Microservices Microservices on Amazon ECS Containers @ Wrapp, presented by Jude D´Souza
  3. 3. What are Microservices?
  4. 4. What are Microservices? “is a software architecture style in which complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs. These services are small, highly decoupled and focus on doing a small task, facilitating a modular approach to system-building.” - Wikipedia https://en.wikipedia.org/wiki/Microservices
  5. 5. Monolithic vs. SOA vs. Microservices SOA Coarse- grained Microservices Fine-grained Monolithic Single unit
  6. 6. Order UI User UI Shipping UI Order Service User Service Shipping Service Data Access Monolithic Architecture
  7. 7. Monolithic Architecture – Scaling
  8. 8. Order UI User UI Shipping UI Order Service User Service Shipping Service Microservices Architecture
  9. 9. Order UI User UI UI Order Service Service Shipping Service Order UI Order UI User UI UIShipping UI Order ServiceOrder Service Service Service Service Service User Service Shipping Service Microservices Architecture – Scaling
  10. 10. What are Microservices Challenges? Resource and state management Data management Monitoring Service discovery Deployment
  11. 11. Containers are Natural for Services Simple to model Any app, any language Image is the version Test & deploy same artifact Stateless servers decrease change risk
  12. 12. What are containers? Self contained Process isolation OS virtualization Automation Server Guest OS Bins/Libs Bins/Libs App2App1
  13. 13. Container advantages Portable Server Guest OS Bins/Libs Bins/Libs App2App1
  14. 14. Container advantages Fast Server Guest OS Bins/Libs Bins/Libs App2App1
  15. 15. Container advantages Efficient Server Guest OS Bins/Libs Bins/Libs App2App1
  16. 16. Managing One Host is Straightforward Server Guest OS Bins/Libs Bins/Libs App2App1
  17. 17. Managing a Fleet is Hard Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS AZ 1 AZ 2 AZ 3
  18. 18. What are Microservices Challenges? Resource and state management Data management Monitoring Service discovery Deployment
  19. 19. Amazon EC2 Container Service
  20. 20. Scalable
  21. 21. Designed for use with other AWS services Elastic Load Balancing Amazon Elastic Block Store Amazon Virtual Private Cloud AWS Identity and Access Management AWS CloudTrail
  22. 22. What are Microservices Challenges? Resource and state management Data management Monitoring Service discovery Deployment
  23. 23. Scheduling Containers on ECS Batch jobs ECS task scheduler Run tasks once Batch jobs RunTask (random) StartTask (placed) Long-running apps ECS service scheduler Health management Scale-up and scale-down Zone-aware Grouped containers ELB (optional)
  24. 24. Scheduling Containers: Long-running App Optionally run your service behind a load balancer. ELB currently supports a fixed relationship between the load balancer port and the container instance port. If a task fails the ELB health check, the task is killed and restarted (until service reaches desired capacity).
  25. 25. Scheduling Containers: Long-running app Deploy using the least space: minimumHealthyPercent = 50%, maximumPercent = 100%
  26. 26. Scheduling Containers: Long-running App Deploy quickly without reducing service capacity: minimumHealthyPercent = 100%, maximumPercent = 200%
  27. 27. Scheduling Containers: Long-running App Blue-Green Deployments • Define two ECS services • Each service is associated w/ ELB • Both ELBs in Route 53 record set with weighted routing policy, 100% Primary, 0% Secondary • Deploy to blue or green service and switch weights TaskTask Route 53 record set with weighted routing policy 0% 100%
  28. 28. What are Microservices Challenges? Resource and state management Data management Monitoring Service discovery Deployment
  29. 29. Monitoring with Amazon CloudWatch Metric data sent to CloudWatch in 1-minute periods and recorded for a period of two weeks Available metrics: CPUReservation, MemoryReservation, CPUUtilization, MemoryUtilization Available dimensions: ClusterName, ServiceName
  30. 30. Monitoring with Amazon CloudWatch
  31. 31. Monitoring with Amazon CloudWatch Use the CloudWatch monitoring scripts to monitor additional metrics: e.g., disk space # Edit crontab > crontab -e # Add command to report disk space utilization to CloudWatch every five minutes */5 * * * * <path_to>/mon-put-instance-data.pl --disk-space-util --disk-space-used --disk- space-avail --disk-path=/ --from-cron
  32. 32. Monitoring Amazon ECS with Datadog
  33. 33. Monitoring Amazon ECS with Sysdig Cloud
  34. 34. Containers @ Wrapp Powered by AWS EC2 Container Service Jude D’Souza Solutions Architect @ Wrapp Email: jude@wrapp.com
  35. 35. About Me … q Jude D’Souza – Stockholm, Sweden ß Karachi, Pakistan jude@wrapp.com q MS – Distributed Systems KTH Royal Institute of Technology, Stockholm, Sweden q Solutions Architect @ Wrapp www.wrapp.com q Passion for building large scale distributed systems
  36. 36. About Wrapp … Wrapp delivers personal offers and rewards based on where you shop, dine and the brands you follow. q Microservice Architecture q Infrastructure q 13 ECS Clusters q 50+ Instances q 90+ Services q Technology Stack q Operating System - Ubuntu q Consul, Registrator, AWS(EC2, ECS, Lambda), Docker, HAProxy q Monitoring – Sensu, Riemann q Logs - Logentries and Sumo q Languages - Go, Python, Java
  37. 37. Agenda § Architecture before ECS – Service Orchestration – Service Discovery – Service Deployments – Service Availability & Scalability – Container Registry § Transition towards ECS – Microservice Architecture @ Wrapp § Why ECS? § Next Challenges § QA
  38. 38. How we did it … Before ECS (1/3) Service Orchestration q Via runlist configs On instance boot: q Determine autoscaling group q Pull runlist for autoscaling group q Run containers specified in config q Add these to supervisor misc ops api misc Autoscaling group ops Autoscaling group api Autoscaling group PULL RUNLIST CONFIGS Sample Runlist configuration file ü Service Orchestration Service Deployments Service Availability & Scalability Container Registry
  39. 39. Service Deployments q Find instances where service is running (via serf) q SSH into these instances q Do rolling deploy (Terminate container, Launch container with new version) mis c ops api misc Autoscaling group - misc-i-abc 10.0.0.1 rewards/http/9090 - misc-i-def 10.0.1.1 rewards/http/9090 - misc-i-ghi 10.0.2.1 rewards/http/9090 - .... - .... Sample serf data for services SERF CLUSTER ops Autoscaling group api Autoscaling group How we did it … Before ECS (2/3) ü Service Orchestration ü Service Deployments Service Availability & Scalability Container Registry
  40. 40. Service Availability q Supervisord Service Scalability q Service itself can't scale q Scale only by scaling autoscaling group How we did it … Before ECS (3/3) Container Registry q Hosting our own container registry q Backed by s3 misc ops api misc Autoscaling group SERF CLUSTER ops Autoscaling group api Autoscaling group misc ü Service Orchestration ü Service Deployments ü Service Availability & Scalability ü Container Registry
  41. 41. Service Orchestration q Runlists q Supervisor Service Deployments q Supervisor Service Availability q Supervisor Service Scalability q EC2 Autoscaling Container Registry q Self-hosting Service Discovery q Serf q HAProxy Service Discovery q Consul q Registrator q HAProxy Service Orchestration Service Deployments Service Availability Service Scalability (with lambda) AWS Container Registry + Service stats (cpu + memory) + Service monitoring + Service Provisioning Transition to ECS
  42. 42. Wrapp Microservice Architecture PUBLIC SUBNE T PRIVATE SUBNET Elastic Load Balancer api api api Microservices ZONE A ZONE B ZONE C api.wrapp.com DNS
  43. 43. Opsweb – Tooling around controlling our Infrastructure Cluster Overview
  44. 44. Opsweb – Tooling around controlling our Infrastructure Services Overview
  45. 45. Service Scalability (Concept from Amazon) mis c misc Autoscaling group Scaling Amazon ECS Services Automatically Using Amazon CloudWatch and AWS Lambda: https://aws.amazon.com/blogs/compute/scaling-amazon-ecs-services-automatically-using-amazon-cloudwatch-and-aws-lambda/ CloudWatch Alarm SNS Topic AWS Lambda EC2 Container Service ECS Instances Trigger CloudWatch alarm from ECS Metrics (CPU/ Memory) Trigger Lambda function from SNS Topic +1 DesiredCount in Service
  46. 46. Service (CPU/ Memory) Provisioning api emailsauth rewards notifications eventbus Typical ECS Instance ü Memory allocation by Service ü Allocate CPU Units by Service ü Efficient Resource utilization (Container placement)
  47. 47. Why ECS? q Our Experience q Generally good service (ECS is oK) q Great support q We don’t need to host these services ourselves on AWS q Always improving to make their services better and better ü Supports built-in service discovery, loadbalancing and routing ü Weave Flux: Supports built-in service discovery, loadbalancing and routing ü Docker Swarm: Supports orchestrating containers ü Fleet - Container Orchestration
  48. 48. api 8080 Consul server-b Consul server-a Consul server-c users 8082 auth 8081 users 8082 Registrator Registrator Registrator Consul Agent Consul Agent Consul Agent Key/value store Service catalog 10.0.0.1 10.0.0.2 10.0.0.3 Service Discovery @ Wrapp (1/2) HA Consul Cluster
  49. 49. Next Challenges (1/2) – Short comings perceived … Built-in (like kube-proxy, weave-proxy) ? q Service Discovery q Service Routing q Service Load balancing Approach 1: ELB Per service? q Costly -90+ services, 70+ ELBs Approach 2: Single ELB for all services? ü Service Routing works ü Service Load balancing works q No health check per service q No way to return HTTP 503 (but possible via haproxy http check option) Benefits: q Removes tools like Consul and Registrator Elastic Load Balancer 8080 8081 8082 ECS Boto3: create_service() Route53: ELB.dns = elb-services frontend users-in bind users:80 mode http default_backend users backend users server users elb-services:8080 … #### >> curl users/ping SINGLE ELB FOR MULTIPLE SERVICES
  50. 50. Next Challenges (2/2) – Short comings perceived … Ø Cron scheduling q Setup ecs tasks to run on cron schedule Ø ECS Event stream q Actionables on certain events (when deploy completes) Ø ECS Container Logging Ø Optionally Integrate ECS Cluster with EC2 Autoscaling Group
  51. 51. Summary q The Wrapp Architecture before ECS q Transition to ECS q Service Discovery @ Wrapp with Consul, Registrator and HAProxy q Our perceived shortcomings from ECS q Built-in Service Discovery q Cron scheduling on Tasks q Event stream
  52. 52. What are Microservices Challenges? Resource and state management Data management Monitoring Service discovery Deployment
  53. 53. Continuous Delivery to ECS with Jenkins 4. Push image to Docker registry 2. Build image from sources 3. Run test on image 1. Code push triggers build 5. Update service 6. Pull image
  54. 54. ECS CI/CD Partners
  55. 55. Continuous Delivery to ECS with Shippable
  56. 56. Thank you!

×