The document discusses Amazon CloudWatch Container Insights, which provides visibility into containerized applications deployed on Amazon ECS and EKS. It describes how Container Insights collects metrics from containers and sends them to CloudWatch, allowing users to monitor tasks and troubleshoot performance issues using Logs Insights and X-Ray. It also provides instructions on enabling Container Insights for ECS using IAM policies or the AWS CLI.

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Webinar
https://amzn.to/JPWebinar https://amzn.to/JPArchive
Amazon CloudWatch Container Insights
[AWS Black Belt Online Seminar]
Solutions Architect
2019/11/27

2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
( )
•
•
AWS LambdaAWS Fargate

3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Black Belt Online Seminar
•
• Q&A blog
•
① 吹き出しをクリック
② 質問を入力
③ Sendをクリック
Twitter
#awsblackbelt

4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• 2019 11 27
AWS (http://aws.amazon.com)
• AWS
AWS
•
• AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to
change in accordance with the AWS Customer Agreement available at
http://aws.amazon.com/agreement/. Any pricing information included in this document is provided
only as an estimate of usage charges for AWS services based on certain information that you
have provided. Monthly charges will be based on your actual use of AWS services, and may vary
from the estimates provided.

5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• :
• Production
•
• :
•
• Container Insights

6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•
•
• Amazon ECS
• Container Insights
• Container Insights
• Container Insights
•
•
•
•
•

7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•
•
• Amazon ECS
• Container Insights
• Container Insights
• Container Insights
•
•
•
•
•

8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•
•
..etc

10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•
Amazon ECS
Amazon Elastic Container
Service
Amazon Elastic
Kubernetes Service
Kubernetes
on
Amazon EC2

12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon ECS
グローバル展開
強力なスケジュールエンジン
オートスケーリング
CloudWatch 連携
(ログ/メトリクス/イベント)
ロードバランサー
コンテナレベルの
ネットワーク構成
高度なタスク配置
戦略
他のAWSサービスと
の連携
ECS CLI
Linux & Windows
コントロールプレーンとして提供
Amazon ECS

13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
ECS
• コンテナ(群)の実行単位
• タスクおよび各コンテナのCPUとメモリ
上限を指定し、それを元にスケジュールさ
れる
• 指定されたタスク数の維持
• ELBとの連携
• メトリクスに応じたオートスケール
• コンテナ実行環境である論理的なグループ
Elastic Load
Balancing

14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights

15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• CloudWatch
•
)
Container Insights
A B C Amazon CloudWatch
Logs
CPU 50%
CPU : 25% CPU : 99% CPU : 25%
Task

16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• ECS
CloudWatch
•
App
Amazon CloudWatch
ECS ECS ECS
Container Insights
App App

17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•
•
• Amazon ECS
• Container Insights
• Container Insights
• Container Insights
•
•
•
•
•

18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights

19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon CloudWatch Container Insights
•
CloudWatch
• CloudWatch
• Container Insights
• AWS Amazon
ECS Amazon EKS Amazon EC2 Kubernetes
※ 2019/11/27 AWS Batch Container Insights

20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon CloudWatch Container Insights
• Amazon CloudWatch
A B C
CPU : 25% CPU : 99% CPU : 25%

21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon CloudWatch Container Insights
• Container Insights

22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon CloudWatch Container Insights
• CloudWatch Logs Insights X-Ray
• Container Insights
Container Insights
Amazon CloudWatch
Logs Insights
AWS X-Ray
•
•
CloudWatch Logs Insights
•
X-Ray

23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights

24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights (1)
• CloudWatch Container
Insights
• Container Insights

25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights (2)
•
• ECS Clusters
• ECS Instances *1
• ECS Services
• ECS Tasks
ECS
*1 Fargate ECS Instances

26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights (2)
•
ECS
*1 Fargate ECS Instances
• ECS Clusters
• ECS Instances *1
• ECS Services
• ECS Tasks

27. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights (3)
•

28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights (3)
•
(container-demo
ecsdemo-frontend )

29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights (4)
.
.
.
• ECS Task

30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• Task performance CloudWatch Logs Insights X-Ray
•
Amazon CloudWatch
Logs Insights
•
•
•
AWS X-Ray
•

31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• Cloudwatch Logs Insights
•
) Exception
Amazon CloudWatch
Logs Insights

32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• CloudWatch Logs Insights
•
)
Amazon CloudWatch
Logs Insights

33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•
•
Amazon CloudWatch
Logs Insights

34. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon CloudWatch
Logs Insights•
•

35. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
for EKS
https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/Container-Insights-view-metrics.html
◼︎Container Insights
stats pct(container_cpu_usage_total, 50) as CPUPercMedian by kubernetes.container_name
| filter Type="Container"
CPU ( )
stats avg(cluster_failed_node_count) as CountOfNodeFailures | filter Type="Cluster" | sort @timestamp
desc
( )
STATS avg(node_cpu_utilization) as avg_node_cpu_utilization by NodeName | SORT
avg_node_cpu_utilization DESC
CPU ( )
Count by container name: stats count() as countoferrors by kubernetes.container_name | filter
stream="stderr" | sort countoferrors desc
( )
Amazon CloudWatch
Logs Insights

36. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS X-Ray
•
•
• ..etc
• Container Insights X-Ray
X-Ray
App
X-Ray
request
response
AWS X-Ray
AWS X-Ray
) X-Ray
◼◼AWS X-Ray
https://docs.aws.amazon.com/ja_jp/xray/latest/devguide/aws-xray.html
AWS X-Ray
API

37. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights

38. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights ECS
• 2:
Container Insights
• AWS CloudFormation AWS Cloud Development Kit(CDK)
• IAM IAM
Container Insights

39. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• Container Insights
Container Insights ECS
 Container Insights
•
• IAM
• IAM
Container Insights

40. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• Container Insights
Container Insights ECS
 Container Insights AWS CLI
aws ecs put-account-setting-default --name containerInsights --value enabled --region us-
east-1
aws ecs put-account-setting --name containerInsights --value enabled --principal-arn
arn:aws:iam::aws_account_id:user/userName --region us-east-1
aws ecs put-account-setting --name containerInsights --value enabled --principal-arn
arn:aws:iam::aws_account_id:role/roleName --region us-east-1

41. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights ECS
• Container Insights Amazon ECS
• Container Insights AWS CLI
$ aws ecs update-cluster-settings --cluster <myCICluster> --settings
name=containerInsights,value=enabled
※ AWS CLI 1.16.200
https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/deploy-container-insights-ECS.html
◼◼Amazon ECS Container Insights

42. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights ECS
• Container Insights ECS Container
Insights

43. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights EKS Kubernetes on
EC2
Container Insights
https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/Container-Insights-prerequisites.html
◼◼Amazon EKS Kubernetes Container Insights
• EKS / Kubernetes on EC2
 Container Insights
 kubectl kubectl apply
• Kubernetes on EC2
 Kubernetes (RBAC)
 kubelet Webhook
 Docker

44. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights EKS Kubernetes on
EC2
Container Insights
https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/deploy-container-insights-EKS.html
◼◼Amazon EKS Kubernetes Container Insights
•
• CloudWatch CloudWatch DaemonSet
• CloudWatch Logs DaemonSet FluentD
•
• Amazon EKS (
CloudWatch Logs )
• StatsD CloudWatch (StatsD
)

45. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

46. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• Container Insights
CloudWatch Logs
•
• CloudWatch Logs Insights
Amazon CloudWatch
Logs
json
json
json
Container Insights
json

47. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•
https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/Container-Insights-reference-performance-logs-ECS.html
https://docs.aws.amazon.com/ja_jp/AmazonCloudWatch/latest/monitoring/Container-Insights-reference-performance-logs-EKS.html
◼︎Amazon ECS Container Insights
ID
CPU
(MB)
MemoryUtilized / MemoryReserved * 100
= 295 / 400
= 75 % [ ]
.
.
◼︎Amazon EKS Container Insights

48. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• CloudWatch Logs
• ECS /aws/ecs/containerinsights/< >/performance
• EKS /aws/containerInsights/< >/performance
) container-demo cluster container

49. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
CloudWatch Logs Insights
• @message
• CpuUtilezed message.CpuUtilezed
) CpuUtilzed MemoryUtilized

50. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

51. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
1.
ECS

52. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•
•
Nginx Rails
256MiB
256
768MiB
768
250 /
250 /
128 /
128 /
CPU : 1024
: 1024MB
•
512MiB
512
1536MiB
1536
250 /
250 /
128 /
128 /
CPU : 2048
: 2048MB
Nginx Rails
•
)

53. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights
• Container Insights
•
Nginx Rails
680MiB
680
344MiB
344
250 /
250 /
128 /
128 /
Nginx Rails
256MiB
256
768MiB
768
250 /
250 /
128 /
128 /
CPU : 1024
: 1024MB
Container Insights
CPU : 1024
: 1024MB

54. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
1. Container
• Container Insights ECS Tasks Container

55. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
.2 Container
• ECS
container-a 256MB/CPU 128
container-b 256MB/CPU 128
container-a 384MB/CPU 192
container-b 128MB/CPU 64

56. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
3. Container
• Container Insights

57. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
2.

58. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
A
•
•
•

59. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
1.
•
•
• >

60. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
2. CPU
•
•
@message

61. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

62. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container Insights
ECS Container Insights
:
10 Amazon EC2 50 20 5
1
CloudWatch :
8 + (6 * 20 ) + (11
* 5 ) = 183 CloudWatch
183 * 0.30USD = 54.90 USD
CloudWatch Logs :
Amazon ECS 1 13 KB
(13 KB/1024/1024) GB * 183 * 730 = 1.66 GB
1.66 GB * 0.50USD = 0.83USD
: 55.73 USD
https://aws.amazon.com/jp/cloudwatch/pricing/
◼︎AWS CloudWatch

63. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• Container Insights
CloudWatch
• Container Insights
• CloudWatch

64. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Q&A
AWS Japan Blog https://aws.amazon.com/jp/blogs/news/

65. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS AWS
https://amzn.to/JPArchive

66. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
個別技術相談会
•
•

67. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Webinar
https://amzn.to/JPWebinar https://amzn.to/JPArchive