Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NEW LAUNCH! Advanced Task Scheduling with Amazon ECS and Blox

1,189 views

Published on

As customers build and run production microservices architectures based on containers, having powerful tools to manage the placement and scheduling of these workloads is critical. Amazon ECS allows customers to focus on building their application and removes the need for managing the cluster management software entirely. This session will focus on the capabilities of the new ECS task placement engine, the built-in ECS schedulers, and explore how customers can build custom schedulers using the ECS event stream and Blox, a newly announced open source scheduling framework that gives developers additional choice and control in how they manage containers in production.

Published in: Technology

NEW LAUNCH! Advanced Task Scheduling with Amazon ECS and Blox

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Anthony Suarez Head of Engineering, AWS Container Services simplycloudio@ NEW LAUNCH! Advanced Task Scheduling with Amazon ECS and Blox CON307 December 1, 2016
  2. 2. What is Amazon ECS? Scalable Container Management Flexible Container Placement AWS Platform Integration Extensible by Design
  3. 3. Amazon ECS: Under the Hood ALB ALB AZ 1 AZ 2 user / scheduler Scheduler Cluster State Service Placement Engine Event Stream
  4. 4. Task Placement Engine
  5. 5. New Placement Constraints & Attributes Name Example AMI ID attribute:ecs.ami-id == ami-eca289fb Availability Zone attribute:ecs.availability-zone == us-east-1a Instance Type attribute:ecs.instance-type == t2.small Distinct Instances type=“distinctInstance” Custom attribute:stack == prod
  6. 6. Anatomy of Task Placement Cluster Constraints Custom Constraints Placement Strategies Apply Filter Satisfy CPU, memory, and port requirements Filter for location, instance-type, AMI, or custom attribute constraints Identify instances that meet spread or binpack placement strategy Select final container instances for placement
  7. 7. Supported Placement Strategies Binpacking Spread Affinity Distinct Instance
  8. 8. Placement Strategy Chaining Spread tasks across Zones and Binpack within each Zone
  9. 9. New Cluster Query Language
  10. 10. Filtering: Match on Instance Family or Type
  11. 11. Filtering: Match on Availability Zone
  12. 12. Filtering: Match on Multiple Expressions
  13. 13. Filtering: Match on Custom Attributes
  14. 14. Task Placement Examples
  15. 15. g2.2xlarge t2.small g2.2xlarge g2.2xlarge Placement: Targeting Instance Type
  16. 16. g2.2xlarge t2.small t2.micro t2.medium t2.medium t2.small g2.2xlarge t2.small t2.small t2.medium us-east-1aus-east-1d Placement: Targeting Instance Type & Zone
  17. 17. g2.2xlarge t2.small t2.micro t2.medium t2.medium t2.small g2.2xlarge t2.small us-east-1aus-east-1d g2.2xlarge t2.medium t2.micro t2.small us-east-1c Placement: Availability Zone Spread
  18. 18. g2.2xlarge t2.small t2.micro t2.medium t2.medium t2.small g2.2xlarge t2.small us-east-1aus-east-1d g2.2xlarge t2.medium t2.micro t2.small us-east-1c Placement: Spread across Zone and Binpack
  19. 19. g2.2xlarge t2.small t2.micro t2.medium t2.medium t2.small g2.2xlarge t2.small us-east-1aus-east-1d g2.2xlarge t2.medium t2.micro t2.small us-east-1c Placement: Affinity and Anti-Affinity
  20. 20. Running a Service
  21. 21. t2.medium t2.small t2.small us-east-1aus-east-1d t2.medium t2.micro t2.small us-east-1c Placement: Multiple Services on a Cluster
  22. 22. t2.medium g2.2xlarge t2.micro t2.small t2.small t2.small g2.2xlarge t2.small t2.small t2.small g2.2xlarge t2.small Placement: Services – Distinct Instances
  23. 23. Console: Getting Started with Placement
  24. 24. Console: Placement Templates to Get Started
  25. 25. Console: Customizing Placement Strategies
  26. 26. Live Demo
  27. 27. Event Stream
  28. 28. Consuming Real-time Events
  29. 29. Three Steps to Getting Started with Events Step 1: Create CWE Rule Step 2: Create SNS Topic Step 3: Put Events to SNS
  30. 30. Consuming Events for Service Discovery app1-tst  10.1.0.11 db1-tst  10.1.0.14 app2  10.1.0.16 db2  10.1.0.18 my-app  10.1.0.20 websrv1 10.1.0.1 websrv2 10.1.0.2 websrv3 10.1.0.4 app-dev1 10.1.0.9 app-dev2 10.1.0.5 app-dev3 10.1.0.8 db-dev 10.1.0.19
  31. 31. Introducing Blox
  32. 32. What is Blox? Choice Control Developer Experience
  33. 33. Building with Blox scheduler cluster state service
  34. 34. Set Up Blox Locally Step 1: git clone https://github.com/blox/blox.git Step 3: docker-compose up -d Step 4: start using Blox locally Step 2: deploy CloudFormation template to configure ECS event stream and SQS queue
  35. 35. Swagger Spec: Cluster State Service
  36. 36. Swagger Spec: Daemon Scheduler
  37. 37. Daemon Scheduler
  38. 38. t2.small t2.small t2.small Blox: Daemon Scheduler t2.small t2.small t2.small scheduler cluster state service t2.small t2.small t2.small
  39. 39. Step 1: Create an Environment
  40. 40. Step 2: Create a Deployment
  41. 41. Step 3: Add Container Instances
  42. 42. Step 4: Monitoring and Scaling a Cluster
  43. 43. Live Demo
  44. 44. The Blox Vision
  45. 45. Contributing to Blox • Blox is licensed under Apache 2.0 • Open an issue or pull request • Watch our roadmap on GitHub • Check out our Gitter channel
  46. 46. Summary
  47. 47. Summary
  48. 48. Thank you!
  49. 49. Remember to complete your evaluations!

×