Build a highly scalable system with serverless container environment - Fargate 利用 Fargate - 無伺服器的容器環境建置高可用的系統 Level: 200, 中文演講 講師: Jayson Hsieh, Solution Architect, AWS & Alan Hsieh, R&D Manager, Growth Machine 使用無伺服器的 Fargate 來部屬 docker 容器環境，是應用服務面對高擴充性、高可用性需求時很好的選擇，免除了底層虛擬機器的維運工作。在這個 session 之中，我們會聊聊以 Fargate 為主體的無伺服器容器環境如何融入 DevOps，並且聽聽 Growth Machine 如何運營上百個 Fargate 容器來滿足高彈性擴張的需求。 聯繫銷售: https://aws.amazon.com/tw/contact-us/ 與銷售線上聊天: https://pages.awscloud.com/tw-hkt-sales-chat.html

1. S U M M I T
Taipei

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
利用 Fargate - 無伺服器的容器環
境建置高延展性的系統
Jayson Hsieh
Solutions Architect
AWS/Solutions Architecture
S e s s i o n I D
Alan Hsieh
R&D Manager
Growth Machine/R&D

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS Container Services landscape
MANAGEMENT
Deployment, scheduling, scaling &
management of containerized
applications
HOSTING
Where the containers run
IMAGE REGISTRY
Container image repository

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Your containerized
applications
Managed by AWS
No EC2 Instances to provision, scale or manage
Elastic
Scale up & down seamlessly. Pay only for what you use
Integrated
with the AWS ecosystem: VPC Networking, Elastic Load
Balancing, IAM Permissions, CloudWatch and more
AWS Fargate

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Fully managed container environment with AWS Fargate
Bring existing code Production ready Powerful integrations
No changes required of existing
code, works with existing workflows
and microservices built
on Amazon ECS
ISO, PCI, HIPAA, SOC compliant.
Launch ten or tens of thousands
of containers in seconds
with 99.99% SLA
Native AWS integrations for
networking, security, CICD,
monitoring, and tracing
Fargate runs tens of millions of containers for AWS customers every week

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS Fargate
Scheduling and Orchestration
Cluster Manager Placement Engine
Availability zone 1 Availability zone 2 Availability zone 3
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1
Container 1 Container 1

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon ECS Constructs
Cluster
• Resource grouping and isolation
• IAM permissions boundary
Service
• Maintains desired # of
running tasks
• Replaces unhealthy tasks
• Elastic Load Balancing
integration
Task
• Running instance of a task
definition
• One or more containers
Task Definition
• Template used by Amazon ECS
to launch tasks
• Parallels to docker run
parameters
• Defines requirements – e.g.
• CPU/Memory
• Container image(s)
• Logging
• AWS Identity and Access
Management (IAM) role
Container 1
Container 1
Container 1
Container 1Container 1
{ ; }
JSON

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Fargate Task CPU and Memory configurations
50 different CPU/Memory configurations to choose from
CPU Memory
256 (.25 vCPU) 512MB, 1GB, 2GB
512 (.5 vCPU) 1GB, 2GB, 3GB, 4GB
1024 (1 vCPU) 2GB, 3GB, 4GB, 5GB, 6GB, 7GB, 8GB
2048 (2 vCPU) Between 4GB and 16GB in 1GB increments
4096 (4 vCPU) Between 8GB and 30GB in 1GB increments

9. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Networking - Launching a Fargate task into a
VPC 172.31.0.0/16
Subnet
172.31.1.0/24
Internet
Other Entities in VPC
EC2 LB DB etc.
Launch your Fargate Tasks into subnets
Under the hood :
• We create an Elastic Network Interface (ENI)
• The ENI is allocated a private IP from your subnet
• The ENI is attached to your task
• Your task now has a private IP from your subnet!
You can assign public IPs to your tasks
Configure security groups to control inbound & outbound traffic
ENI Fargate
Task

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Observability
“…a measure of how well internal states of a system can be inferred from
knowledge of its external outputs.” – Wikipedia

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Monitoring
Amazon CloudWatch ECS metrics
Service CPU/memory utilization
Amazon ECS metadata endpoint
Query task metadata and access Docker
stats for running tasks
169.254.170.2/v2/metadata
169.254.170.2/v2/stats
Partners
"read": "2018-11-17T01:39:05.558680889Z",
"preread": "2018-11-17T01:39:04.569462567Z",
"num_procs": 0,
"pids_stats": {},
"network": {},
"memory_stats": {
"stats": {
"cache": 6127616,
"mapped_file": 2117632,
"total_inactive_file": 1626112,
"pgpgout": 3056,
"rss": 3981312,
"total_mapped_file": 2117632,
"pgpgin": 5524,
"pgmajfault": 51,
"total_rss": 3981312,
"hierarchical_memory_limit": 536870912,
"total_pgfault": 5865,
"total_active_file": 4501504,
"active_anon": 3981312,
"total_active_anon": 3981312,
"total_pgpgout": 3056,

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Logging
Amazon CloudWatch Logs
Fargate awslogs log driver
STDOUT and STDERR I/O streams
Stream logs to Amazon Kinesis Data
Streams, Amazon Kinesis Data Firehose,
AWS Lambda
Subscription filters
Container 1
logs logs

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Logging
Sidecar container
Fargate provides 4GB ephemeral storage
Amazon Kinesis agent sidecar pattern (below)
Custom logging solutions
logs

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Distributed Tracing
User
AWS X-Ray
Analyze and debug distributed applications
Useful for microservices environments
X-Ray SDK for app instrumentation
X-Ray daemon runs as sidecar
Service
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.ext.flask.middleware import XRayMiddleware
app = Flask(__name__)
xray_recorder.configure(service=‘Microservice’)
XRayMiddleware(app, xray_recorder)

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Distributed Tracing

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Alerting
Amazon CloudWatch
Collects metrics, events, logs produced by
services/apps
Triggers actions based on metric thresholds and
event/log patterns
Service Integration
Amazon SNS
User notifications – email, SMS, mobile app
System-to-system messaging – AWS Lambda,
SQS, HTTP/S endpoint
AWS Lambda
Event-driven compute
Rule Alarm
Traditional
server
metrics
events
logs
AWS Partner /
Other Consumer

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
CI/CD for Fargate
Amazon EC2
Container Registry
AWS CodeCommit
AWS Fargate
AWS CodeBuild
AWS CodePipeline
Github Jenkins

19. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Exchange User Behavior
• Need very timely price and quantity
• Lot of actions trigger by event
• Long time and multi-page to watch the price
• Massive of limit orders

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Exchange User Behavior
• Need very timely price and quantity
→ Update 5 times per sec
• Lot of actions trigger by event
→ Massive of instant traffic
• Long time and multi-page to watch the price
→ Multiple growth traffics
• Massive of limit orders
→ High concurrent transactions

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
System Requirement
Handle high concurrency and scale fast, massive

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
• Service Architecture
• Why Fargate
• How to work and scale
• Maintain and monitor service
• Other services

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Architecture - Service Layer

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Architecture - Service Layer

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Solution Comparison
ECS EKS FARGATE
Management Server + Cluster Server + Cluster Serverless
Support Windows + Linux Windows + Linux Linux
Integration AWS DevOps tools
Cross-platform
K8S sync
AWS DevOps tools
Auto Scaling Container + Server Container + Server Container
Pricing Server Server + Cluster CPU / Memory unit
Others Support spot instance Community resources Higher security

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Why Fargate
We need a stable and massive auto scaling
service without maintaining server

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Work With Fargate

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Scalable Components

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How To Scale
• Set policies according to best Response Time for each service
• Set Min, Desired, Max number of tasks
• Set target tracking by CPU Utilization, Memory Utilization, Request Count
• Set step tracking by Unhealthy Count
• Set number of tasks in scheduled tasks feature by fixed time

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Renew Task
• Automatically renew tasks at a specific time
• Set [ Force new deployment ] to renew service

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
CloudWatch - Log Insight
• Set log export in ECS Task Definition
• View all logs across tasks
• Filter some keyword in logs
• Visualize log events over time

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Distributed Tracking System
• Trace all the requests for all the services
• Easier to debug and find the issues
• Analysis and tune performance
• Visualize the service relationship and request flow

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
XRAY - Service Map
• Find the node which has problem at first glance

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
XRAY - Traces
• Record segment flow, time cost, annotations, exceptions

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
XRAY - Analytics
• Analysis requests with different periods and figure out the issues

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Hystrix
• Circuit breaker by Netflix
• Protect service with fault tolerance
• Timeout / Breaker / Fallback / Healthy Check
• Use to every external request
• Monitor healthy status of request
• Monitor concurrent loading

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Hystrix Flow

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Hystrix Dashboard

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Deploy - AWS DevOps Tools

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Aurora DB
• Storage auto scaling
• Read replica auto scaling
• Automatically failover
• Support cross-region replica
• Serverless type
• Parallel query type

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Experience
• Understand user behavior and what system we need to provide is very important
• Always plan and use easily scalable solutions
• Periodically analysis service performance to reset autoscaling policies
• Use centralized and analyzed log services in massive of containers and micro services
• Planning ahead log data which we may need will solve problem more quickly

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Alan Hsieh
alanhsieh@boyu66.net
Jayson Hsieh
hsiej@amazon.com

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.