More Related Content
Similar to 利用Fargate無伺服器的容器環境建置高可用的系統
Similar to 利用Fargate無伺服器的容器環境建置高可用的系統 (20)
More from Amazon Web Services
More from Amazon Web Services (20)
利用Fargate無伺服器的容器環境建置高可用的系統
- 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
利用 Fargate - 無伺服器的容器環
境建置高延展性的系統
Jayson Hsieh
Solutions Architect
AWS/Solutions Architecture
S e s s i o n I D
Alan Hsieh
R&D Manager
Growth Machine/R&D
- 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Container Services landscape
MANAGEMENT
Deployment, scheduling, scaling &
management of containerized
applications
HOSTING
Where the containers run
IMAGE REGISTRY
Container image repository
- 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Your containerized
applications
Managed by AWS
No EC2 Instances to provision, scale or manage
Elastic
Scale up & down seamlessly. Pay only for what you use
Integrated
with the AWS ecosystem: VPC Networking, Elastic Load
Balancing, IAM Permissions, CloudWatch and more
AWS Fargate
- 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fully managed container environment with AWS Fargate
Bring existing code Production ready Powerful integrations
No changes required of existing
code, works with existing
workflows and microservices
built on Amazon ECS
ISO, PCI, HIPAA, SOC compliant.
Launch ten or tens of thousands
of containers in seconds
with 99.99% SLA
Native AWS integrations for
networking, security, CICD,
monitoring, and tracing
Fargate runs tens of millions of containers for AWS customers every week
- 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Fargate
Scheduling and Orchestration
Cluster Manager Placement Engine
Availability zone 1 Availability zone 2 Availability zone 3
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
- 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon ECS Constructs
Cluster
• Resource grouping and isolation
• IAM permissions boundary
Service
• Maintains desired # of
running tasks
• Replaces unhealthy tasks
• Elastic Load Balancing
integration
Task
• Running instance of a task
definition
• One or more containers
Task Definition
• Template used by Amazon ECS
to launch tasks
• Parallels to docker run
parameters
• Defines requirements – e.g.
• CPU/Memory
• Container image(s)
• Logging
• AWS Identity and Access
Management (IAM) role
Container 1
Container 1
Container 1
Container 1Container 1
{ ; }
JSON
- 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fargate Task CPU and Memory configurations
50 different CPU/Memory configurations to choose from
CPU Memory
256 (.25 vCPU) 512MB, 1GB, 2GB
512 (.5 vCPU) 1GB, 2GB, 3GB, 4GB
1024 (1 vCPU) 2GB, 3GB, 4GB, 5GB, 6GB, 7GB, 8GB
2048 (2 vCPU) Between 4GB and 16GB in 1GB increments
4096 (4 vCPU) Between 8GB and 30GB in 1GB increments
- 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Networking - Launching a Fargate task into a VPC
172.31.0.0/16
Subnet
172.31.1.0/24
Internet
Other Entities in VPC
EC2 LB DB etc.
Launch your Fargate Tasks into subnets
Under the hood :
• We create an Elastic Network Interface (ENI)
• The ENI is allocated a private IP from your subnet
• The ENI is attached to your task
• Your task now has a private IP from your subnet!
You can assign public IPs to your tasks
Configure security groups to control inbound & outbound traffic
ENI Fargate
Task
- 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Observability
“…a measure of how well internal states of a system can be inferred from
knowledge of its external outputs.” – Wikipedia
- 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Monitoring
Amazon CloudWatch ECS metrics
Service CPU/memory utilization
Amazon ECS metadata endpoint
Query task metadata and access Docker
stats for running tasks
169.254.170.2/v2/metadata
169.254.170.2/v2/stats
Partners
"read": "2018-11-17T01:39:05.558680889Z",
"preread": "2018-11-17T01:39:04.569462567Z",
"num_procs": 0,
"pids_stats": {},
"network": {},
"memory_stats": {
"stats": {
"cache": 6127616,
"mapped_file": 2117632,
"total_inactive_file": 1626112,
"pgpgout": 3056,
"rss": 3981312,
"total_mapped_file": 2117632,
"pgpgin": 5524,
"pgmajfault": 51,
"total_rss": 3981312,
"hierarchical_memory_limit": 536870912,
"total_pgfault": 5865,
"total_active_file": 4501504,
"active_anon": 3981312,
"total_active_anon": 3981312,
"total_pgpgout": 3056,
- 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logging
Amazon CloudWatch Logs
Fargate awslogs log driver
STDOUT and STDERR I/O streams
Stream logs to Amazon Kinesis Data
Streams, Amazon Kinesis Data Firehose, AWS
Lambda
Subscription filters
Container 1
logs logs
- 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logging
Sidecar container
Fargate provides 4GB ephemeral storage
Amazon Kinesis agent sidecar pattern (below)
Custom logging solutions
logs
- 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distributed Tracing
AWS X-Ray
Analyze and debug distributed applications
Useful for microservices environments
X-Ray SDK for app instrumentation
X-Ray daemon runs as sidecar
User Service
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.ext.flask.middleware import XRayMiddleware
app = Flask(__name__)
xray_recorder.configure(service=‘Microservice’)
XRayMiddleware(app, xray_recorder)
- 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distributed Tracing
- 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Alerting
Amazon CloudWatch
Collects metrics, events, logs produced by
services/apps
Triggers actions based on metric thresholds and
event/log patterns
Service Integration
Amazon SNS
User notifications – email, SMS, mobile app
System-to-system messaging – AWS Lambda,
SQS, HTTP/S endpoint
AWS Lambda
Event-driven compute
Rule Alarm
Traditional
server
metrics
events
logs
AWS Partner /
Other Consumer
- 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CI/CD for Fargate
Amazon EC2
Container Registry
AWS CodeCommit
AWS Fargate
AWS CodeBuild
AWS CodePipeline
Github Jenkins
- 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customer Reference –
Growth Machine
Alan Hsieh / R&D Manager
- 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Exchange User Behavior
• Need very timely price and quantity
• Lot of actions trigger by event
• Long time and multi-page to watch the price
• Massive of limit orders
- 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Exchange User Behavior
• Need very timely price and quantity
→ Update 5 times per sec
• Lot of actions trigger by event
→ Massive of instant traffic
• Long time and multi-page to watch the price
→ Multiple growth traffics
• Massive of limit orders
→ High concurrent transactions
- 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
System Requirement
Handle high concurrency and scale fast, massive
- 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
• Service Architecture
• Why Fargate
• How to work and scale
• Maintain and monitor service
• Other services
- 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Architecture - Service Layer
- 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Architecture - Service Layer
- 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Solution Comparison
ECS EKS FARGATE
Management Server + Cluster Server + Cluster Serverless
Support Windows + Linux Windows + Linux Linux
Integration AWS DevOps tools
Cross-platform
K8S sync
AWS DevOps tools
Auto Scaling Container + Server Container + Server Container
Pricing Server Server + Cluster CPU / Memory unit
Others Support spot instance Community resources Higher security
- 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Why Fargate
We need a stable and massive auto scaling
service without maintaining server
- 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Work With Fargate
- 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Scalable Components
- 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How To Scale
• Set policies according to best Response Time for each service
• Set Min, Desired, Max number of tasks
• Set target tracking by CPU Utilization, Memory Utilization, Request Count
• Set step tracking by Unhealthy Count
• Set number of tasks in scheduled tasks feature by fixed time
- 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Renew Task
• Automatically renew tasks at a specific time
• Set [ Force new deployment ] to renew service
- 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
CloudWatch - Log Insight
• Set log export in ECS Task Definition
• View all logs across tasks
• Filter some keyword in logs
• Visualize log events over time
- 33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Distributed Tracking System
• Trace all the requests for all the services
• Easier to debug and find the issues
• Analysis and tune performance
• Visualize the service relationship and request flow
- 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
XRAY - Service Map
• Find the node which has problem at first glance
- 35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
XRAY - Traces
• Record segment flow, time cost, annotations, exceptions
- 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
XRAY - Analytics
• Analysis requests with different periods and figure out the issues
- 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Hystrix
• Circuit breaker by Netflix
• Protect service with fault tolerance
• Timeout / Breaker / Fallback / Healthy Check
• Use to every external request
• Monitor healthy status of request
• Monitor concurrent loading
- 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Hystrix Flow
- 39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Hystrix Dashboard
- 40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Deploy - AWS DevOps Tools
- 41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Aurora DB
• Storage auto scaling
• Read replica auto scaling
• Automatically failover
• Support cross-region replica
• Serverless type
• Parallel query type
- 42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Experience
• Understand user behavior and what system we need to provide is very important
• Always plan and use easily scalable solutions
• Periodically analysis service performance to reset autoscaling policies
• Use centralized and analyzed log services in massive of containers and micro services
• Planning ahead log data which we may need will solve problem more quickly
- 43. Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Alan Hsieh
alanhsieh@boyu66.net
Jayson Hsieh
hsiej@amazon.com