This document discusses service discovery patterns and considerations. It summarizes different approaches to service discovery including using load balancers, DNS, key-value stores, and service meshes. It also provides a case study of how Kaplan Test Prep moved to a microservices architecture on AWS and used service discovery to reduce costs and improve developer productivity. Specifically, they flipped their system-to-service ratio from 3-1 to 1-15, enabled self-service deployments through CI/CD pipelines, and reduced infrastructure costs by 80% for migrated applications.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Patterns and Considerations
in Service Discovery
AWS re:INVENT
Shubha Rao, Sr. Product Manager, Amazon ECS
Roven Drabo, Head of Cloud Operations, Kaplan Test Prep
November 29, 2017

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Application communication is evolving
Functional calls
Find endpoints,
then connect
Across the room
Dynamic name,
number, and location
Across cities 1:1
Known endpoints, APIs

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Developers need to connect micro services
Build apps
invoking other
services by name
Ensure that service
name resolves to
correct IP/port
DEV OPS

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What is Service Discovery?
“Where is Service X?”
Friendly name -> IP + port
E.g., app: {10.0.4.5:8080, 10.0.4.6:8080 }

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why is it non trivial?
Dynamic by design:
Number of containers and instances
Auto assigned IP addresses and ports
Placement, scheduling, scaling
Deployments and upgrades
Health and connectivity

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Decision criteria
Service registry
Discovery mechanism
Server side or client side
Registration mechanism
Third party or application framework

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
E.g.,
Load Balancer
Benefits
Client is simpler
Drawbacks
Install, manage
Availability, capacity
More hops
Discovery on the server side
Client
Server
Server
Server
Request Request
LB +
Service
Registry
Register
Register

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Discovery on the client side
E.g.,
Route 53 Based
Benefits
Fewer hops
Drawbacks
Client must be registry aware
Client implements discovery logic
Client
Server
Server
Server
Service
Registry
Request
Register
Register
Register
Query

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Registration by third party
E.g.,
Consul, Etcd and Zookeeper
Benefits
Registrar registers, unregisters
and performs health checks
Service less complex
Drawbacks
State may not reflect whether service can
handle requests, only active or unavailable
Install, configure and manage other
component, unless it is a part of infrastructure
Server
Server
Server
Service
Registry
Update Start, stop,
health change
Registrar/
service manager

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Registration by service language framework
E.g.,
Microservice chassis frameworks
Java: Spring Boot, Spring Cloud, Dropwizard
Go: Gizmo, Micro and Go kit
Benefits
Potentially more states: e.g., STARTING, AVAILABLE, etc.
Drawbacks
Couples the service to the Service Registry
Must implement registration logic in each programming
language/framework
Service/frameworks often lack the
self-awareness to unregister
Server
Server
Server
Service
Registry
Start, stop,
health change

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Patterns for Service Discovery
Load Balancers Service MeshDNS Key/Value store

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1. Using Load Balancers
Server side service discovery
Service Registry = Load Balancer
target groups
Registry update done by ECS scheduler
Similar architecture with Elastic
Load Balancer, Network Load Balancer,
or Application Load Balancer
Load
Balancer
ECS
Instance BInstance A

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2. Using DNS, e.g., ECS + Route 53 (New!)
Cluster: myapp
App
10.0.6.5:8080
Web
10.0.8.6:8080
App
10.0.4.5:8080
Web
10.0.3.6:8080
Namespace: myapp.local
Service: web.myapp.local
10.0.4.5:8080,10.0.3.6:8080
Service: app.myapp.local
10.0.6.5:8080,10.0.8.6:8080
Client side service discovery
Service Registry = Route53 DNS Server
Registry update done by ECS scheduler
Managed, high availability,
high scale, extensible

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
3. Using key-value stores, e.g., Consul
Service
registry
Client side Service Discovery
Service Registry = Route53 DNS Server
Registry update done by ECS scheduler
Managed, high availability, high scale, extensible
ECS Services
Consul-
agent
Registrator
Weather
Stock-price
Portal
ECS Services
Registrator
Consul-
agent
ECS Instance ECS Instance
ECS Cluster
ECS Instance
Consul-
agent

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4. Using Service Mesh e.g., Linkerd, Envoy
Service Mesh handles service discovery
Linkerd/Envoy proxy maintain
service registry
Registry updates done by third party tool
like Consul

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Microservices at Kaplan Test Prep
Roven Drabo, Head of Cloud Operations
Customer case study

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kaplan Test Prep
Offers preparation for more
than 90 standardized tests,
including entrance exams
for secondary school,
college, and graduate school
Professional
Licensing exams
Provides private
tutoring and graduate
admissions consulting

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Our business challenges
Reduce time to
market (TTM) for new
products and features
Reduce IT
infrastructure and
operational costs
Improve reliability and
scalability of our
core applications
Improve engineers
overall satisfaction
and level of happiness

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Architecture and engineering challenges
Large systems-to-services ratio; more systems equals more
operational overhead
Manual release process, no CI/CD pipelines, and complex scripts to maintain
Legacy monolithic apps and web services needed to be transformed
into infrastructure as code and rearchitected along the way

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Infrastructure requirements and needs
Needs
Need for an orchestration which would fit our needs
CloudFormation to automate and deploy AWS services
Consul Key/Value Store as a global state of the
microservices
NGINX for Routing, Customization and Proxying
Requirements
Easy Cluster Management
Container Management/Scheduling
Auto healing/scaling
AWS Integration

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SSDR high level architecture

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Build and deploy in < 2 minutes
udeploy qa create –s
/mystack/myfiles.yaml
CFML CloudFormation
template
SSDR
Platform

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
udeploy example.yaml

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
udeploy CLI deployment example

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Creates CloudFormation stack and substacks

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Service up and running

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
example.yaml with unencrypted password

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Password encrypted using udeploy

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
example.yaml with encrypted password

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Summary
Flipped the ratio of systems-to-services from 3-1 to 1-15
Code deployments now require 1 person and take on average < than 5 mins
Enabled self service for developers to push code to quality assurance, staging
and production through a CI/CD pipeline
Operational workload has diminished and improve release cycles
% of code covered by automated testing grew from 50% to 80%
Infrastructure savings for migrated apps average 80%

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THANK YOU!