The document discusses service mesh and AWS App Mesh. It provides an overview of the challenges of microservices architectures like control over communication between services and lack of visibility. It then describes how a service mesh like AWS App Mesh addresses these challenges by providing a control plane that configures proxies for observability, traffic management and resiliency without requiring changes to application code. It presents AWS App Mesh as a fully managed service mesh that works across compute platforms like ECS, EKS and EC2.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
마이크로 서비스를 위한
AWS Cloud Map & App Mesh
Saeho Kim

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Complexity of modern architectures
• Wide variety of resources
• Complexity grows exponentially
• Multiple versions and stages coexist
• Infrastructure scales dynamically
• Unhealthy resources are replaced

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Complexity of modern architectures
Service Discovery
Finding the location of a service provider
myapp: {10.24.34.5:8080, 10.24.34.6:8080}
mylogs: {S3bucket1, S3bucket2}

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Server-side service discovery pattern
• Connections are proxied
• Discovery is abstracted away
• Availability and capacity impact
• Additional latency
Client
Service Provider
Service Provider
Service Provider
Request Request
LB +
Service
Registry
Register
Register

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Client-side service discovery pattern
• Clients connect directly to providers
• Fewer components in the system
• Clients must be registry-aware
• Client-side load balancing
Client
Service Provider
Service Provider
Service Provider
Service
Registry
Request
Register
Register
Register
Query

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build the dynamic map of your cloud

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Registry for all cloud resources
webserver running
on EKS
backend component
running on ECS
shared event registration
service on Lambda
shared logs on S3
payment integration
running on EC2 in
ASG
payment DB on RDS

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Registry for all cloud resources
frontend.app.com backend.app.com
eventprocessor.shared logs.shared
payments.app.com payments-db.app.com

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud Map registry
• Namespace
• Service
• Service Instance
cloudmapdemo.com
backend
Name = backend
DNS record = A
TTL = 60 sec
Health Check = Yes
Instance-1
172.10.0.1
Instance-2
172.10.0.2
Instance-3
172.10.0.3

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Register resources for API + DNS discovery mode
1. aws servicediscovery create-public-dns-namespace --name cloudmapdemo.com
2. aws servicediscovery create-service --name frontend
--dns-config “NamespaceId=%namespace_id%, DnsRecords=[{Type=A, TTL=60}]”
3. aws servicediscovery register-instance --service-id %service_id% --instance-id %id%
--attributes
AWS_INSTANCE_IPV4=52.89.144.60,
stage=beta,
version=1.0,
ready=yes

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Register any cloud resource for API discovery
1. aws servicediscovery create-http-namespace --name shared
2. aws servicediscovery create-service --name logs --namespace-id %namespace_id%
3. aws servicediscovery register-instance --service-id %service_id% --instance-id %id%
--attributes
ARN=arn:aws:s3:::cloudmapdemoservicelogsbeta1,
stage=beta,
shard=s_1,
read_only=no,
path=/mylogs

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Secure name resolution via API calls
aws servicediscovery discover-instances --namespace-name shared --service-name logs
-->
{ "Instances": [
{
"InstanceId": "i1",
"NamespaceName": "shared",
"ServiceName": "logs",
"HealthStatus": "UNKNOWN",
"Attributes": {
"read_only": "no",
"path": "/mylogs",
"shard": "s_1",
"ARN": "arn:aws:s3:::cloudmapdemoservicelogsbeta1",
"stage": "beta”
}
}
]
}

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Name resolution via DNS
dig +short A frontend.cloudmapdemo.com
-->
52.89.144.60
52.26.95.129
34.214.232.177

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attribute-based service discovery

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attribute-based service discovery
Stage: prod

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attribute-based service discovery
Version: 1.0

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attribute-based service discovery
Version: 2.0
Ready: yes

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attribute-based service discovery
Register and discover resources with custom attributes
• Incremental deployments
• Dev / Test / Prod deployments
• Smart traffic routing

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Attribute-based service discovery
1. aws servicediscovery discover-instances --namespace-name cloudmapdemo.com
--service-name frontend --query-parameters ready=yes
-->
{ "Instances": [
{
"InstanceId": "i1",
"NamespaceName": "cloudmapdemo.com",
"ServiceName": "frontend",
"HealthStatus": "UNKNOWN",
"Attributes": {
"ready": "yes",
"AWS_INSTANCE_IPV4": "52.89.144.60",
"version": "1.0",
"stage": "beta”
}
}
]
}

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Handling partial failure
AWS Cloud Map provisions Amazon Route 53 health checks for IP-based
resources
• Unhealthy resources are removed from query responses
• API to check health status - getInstancesHealthStatus

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Handling partial failure
1. aws servicediscovery create-service --name users
--dns-config “NamespaceId=%namespace_id%, DnsRecords=[{Type=A, TTL=60}]”
--health-check-config “Type=TCP, FailureThreshold=3”
2. aws servicediscovery register-instance --service-id %service_id --instance-id
healthy-instance –attributes AWS_INSTANCE_IPV4=52.89.144.60,AWS_INSTANCE_PORT=80
3. aws servicediscovery register-instance --service-id %service_id --instance-id
unhealthy-instance --attributes AWS_INSTANCE_IPV4=88.88.88.88,AWS_INSTANCE_PORT=80

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Handling partial failure
- dig +short A users.cloudmapdemo.com
--> 52.89.144.60
- aws servicediscovery get-instances-health-status --service-id %service_id%
--> {
"Status": {
"healthy-instance": "HEALTHY",
"unhealthy-instance": "UNHEALTHY”
}
}
- aws servicediscovery discover-instances --namespace-name cloudmapdemo.com --service-
name users --health-status UNHEALTHY

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Control traffic with custom health checks
AWS Cloud Map supports custom health checks that reliably and quickly
propagate health information updates
• You control when to start and stop traffic routing
• Create service with HealthCheckCustomConfig
• Use UpdateInstanceCustomHealthCheck API to set status to HEALTHY |
UNHEALTHY

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud Map ecosystem
Amazon ECS Amazon EKSAWS App Mesh

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon ECS Service Discovery
Service discovery via DNS and API
- Additional attributes for ECS tasks
- Smart routing based on locality
- Support for EC2 and Fargate (1.10+)
launch types
AWS_INSTANCE_IPV4
AWS_INSTANCE_PORT
AVAILABILITY_ZONE
REGION
ECS_SERVICE_NAME
ECS_CLUSTER_NAME
EC2_INSTANCE_ID
ECS_TASK_DEFINITION_FAMILY

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubernetes ExternalDNS Connector
frontend service frontend.app.com

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud Map availability
US West (Oregon)
US East (N. Virginia)US West (N. California) Asia Pacific (Tokyo)
Asia Pacific (Sydney)
US East (Ohio)
Asia Pacific (Mumbai)
Asia Pacific (Seoul)
Canada (Central)
Asia Pacific (Singapore)
EU (Ireland)
EU (Frankfrut)
EU (London)

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Challenges with Microservices
Control Over Service to Service Communication
Visibility into Service to Service Communication i.e., Observability
Create a culture of innovation by organizing into small DevOps teams
Ensure trust by automating security and compliance

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is needed
Consistent
communications
management
Complete visibility Failure isolation
and protection
Fine-grained
deployment controls

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
.NET
Go
Go
Django
.NET
Node.js
Node.js
Node.js
Java
GoNode.js
Java

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Monitoring
Degraded state
.NETGo

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Degraded state
Outage
Latency
Time (ms)

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Resiliency patterns
Traffic shaping Rate limiting Circuit breaking Retries Throttling

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Implementation options
Microservice
container
In-process
(SDK)
Out-of-process
(sidecar proxy)
Microservice
container
Proxy
Option 1 Option 2

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Implementation options
Microservice
container
In-process
(SDK)
Out-of-process
(sidecar proxy)
Microservice
Container
Proxy
Option 1 Option 2

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Option 1: In-process resiliency SDK
SDK maintenance
Application code changes
Retrofitting
Unknown dependencies
…
Java
Scala
Node.js
Python
C++
Django
.NET
GO
…
…
MySQL (hosted + Amazon Relational
Database Service (Amazon RDS))
Aurora
Microsoft SQL Server
PostgreSQL (hosted and Amazon
RDS)
Redis
InfluxDB
RabbitMQ
MongoDB
Amazon DynamoDB
Cassandra
…
Languages
Databases

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Implementation options
Microservice
container
In-process
(SDK)
Out-of-process
(sidecar proxy)
Microservice
container
Proxy
Option 1 Option 2

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Option 2: Side-car proxy
Decouple operational logic and SDKs
Microservice
container
Proxy
Amazon ECS task / Kubernetes Pod
Port
8081
Port
8080External traffic

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Option 2: Side-car proxy
Out-of-process and language
independent:
Logging
Tracing
Metrics
Resiliency patterns
Separation of operational and business
logic
Integration with legacy services
However…

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Centralized production-grade configuration
of proxies at scale is difficult

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We need a control plane
Centralized location to manage configuration of proxies at scale
Dynamic configuration reload without redeploying code
Compatibility across different compute platforms
Production-grade and fully managed

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Introducing AWS App Mesh
Service mesh for AWS
Observability and traffic control
Easily export logs, metrics, and traces
Client-side traffic policies—circuit breaking, retries
Routes for deployments
Works across clusters and container services
Amazon ECS
Amazon EKS
Kubernetes on EC2
AWS built and run
Managed control plane
Production-grade

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS App Mesh configures every proxy
Microservice
Proxy

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Easily deliver configuration and receive data
Infra
Operator
Application
Developer Metrics
Intent
Microservice
Proxy

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why AWS App Mesh
Libraries or application code vs. mesh
Overall—migrate to microservices safer and faster
Reduce work required
by developers
Provide operational
controls decoupled
from application logic
Use any language
or platform
Simplify visibility,
troubleshooting, and
deployments

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
App Mesh uses Envoy proxy
OSS project managed by CNCF
Started at Lyft in 2016
Wide community support, numerous integrations
Stable and production-proven

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why AWS App Mesh
vs. building or running your own mesh
No need to spend on
Dev to build and Ops
to maintain
Not tied to application
deployment system
(e.g., container orchestration)
Works across different
compute systems
Gradual migration,
onboard services
one at a time

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why AWS App Mesh
vs. existing control plane solutions
Works across
clusters,
container services
Integrations with AWS
and partner tools
Run by AWS for scale
and stability
Extensible architecture
from OSS base

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Services connect directly

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deployments
B
B’
5%
95%
A

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Traffic controls
Routing options
Service discovery
Retires
Timeouts
Error-code recognition
Routing controls
Access
Quotas
Rate limits
Weights

56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Application observability
+ others
Universal metrics
collection for
a wide range of
monitoring tools

57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
App Mesh Constructs
Mesh
Virtual node
Virtual router and routes
Virtual service
Create and manage these in App
Mesh API, CLI, SDK, or
AWS Management Console
Proxies
Services
Service discovery
Configure and run proxies and
services on Amazon ECS, Fargate,
Amazon EKS, Amazon EC2
Service discovery with
AWS Cloud Map

58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
B
A
Mesh – [sample_app]
Elastic
Load
Balancing
Virtual
node A
Service
discoveryListener Backends
Virtual
node B
Service
discoveryListener Backends
App MeshMicroservices
How it works

59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Virtual node
Virtual node
Service
discovery
BackendsListeners
Virtual node
Logical representation
of runtime services.
Backends
Set of destinations that this node
will communicate with (hostnames)
Service discovery
Describes how its callers locate this node
Listeners
Policies to handle
incoming traffic

60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mesh – [sample_app]
Virtual router
HTTP route
Targets:
Prefix: /
B
B’
Virtual
node A
Service
discovery
Listener Backends
Virtual
node B
Service
discovery
Listener Backends
Virtual
node B’
Service
discovery
Listener Backends
B
B
B’
B’
A
Connecting microservices

61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deployments
B
B’
5%
95%
A

62. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Virtual router
Virtual router
HTTP route
Prefix: /
Targets:
B
B’
Destination’s virtual
router and route
Route B
Destination + weight
Route B’
New service versionB
B
B’
B’
A

63. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Representing your sample_app in AWS App Mesh
A
B
B
C
C
D
D
Mesh – [sample_app]
Service C
Virtual router
Virtual
node C
Service D
Virtual router
Virtual
node D
Service A
Service B
Virtual
router
Virtual
node B

64. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Today, App Mesh is generally available worldwide
Observability and traffic control
Easily export logs, metrics, and traces
Client-side load balancing, routing
AWS container services compatibility
Amazon Elastic Container Service (Amazon ECS)
Amazon Elastic Container Service for Kubernetes (Amazon EKS)
AWS Fargate
EC2 compatibility
Integrate with services running directly on EC2 instances

65. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS App Mesh Availability
US West (Oregon)
US East (N. Virginia)US West (N. California) Asia Pacific (Tokyo)
Asia Pacific (Sydney)
US East (Ohio)
Asia Pacific (Mumbai)
Asia Pacific (Seoul)
Canada (Central)
Asia Pacific (Singapore)
EU (Ireland)
EU (Frankfrut)
EU (London)

66. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Roadmap
Today
Client-side load balancing
Path-based routing
Egress traffic
AWS Cloud Map service discovery
App Mesh console
AWS X-Ray tracing
Envoy supported tracing
CloudWatch Logs, metrics
StatsD, Prometheus metrics
Amazon ECS, Fargate integration
Amazon EKS integration
Amazon EC2 integration
Near term
AWS PrivateLink
AWS Cloud Map selectors
gRPC routing
Header-based routing
Cookie-based routing
Host-based routing
Timeout policy
Retry policy
Circuit breaker policy
End-to-end encryption
Longer term
mTLS
Mesh peering
Global rate limiting
Managed ingress
Other protocols

67. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

68. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Gateway
container
Gateway
http Colorteller
container
Colorteller
httphttp
Gateway
Proxy applies routing rules (e.g., path matching)
Proxy does the load balancing if there is more
than one Colorteller task running
Proxy does metrics, logging, and tracing
Colorteller
Proxy also does metrics, logging, and tracing
http

69. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Next Step
• Product overview
https://aws.amazon.com/cloud-map/
https://aws.amazon.com/app-mesh
• Documentation
https://docs.aws.amazon.com/cloud-map/index.html
https://docs.aws.amazon.com/app-mesh/index.html
• Examples
https://github.com/aws/aws-app-mesh-examples
• Issues, roadmap, beta channel
https://github.com/aws/aws-app-mesh-roadmap