Patterns and Considerations in Service Discovery - Con327 - re:Invent 2017

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Patterns and Considerations
in Service Discovery
Shubha Rao, Sr. Product Manager, Amazon ECS
Roven Drabo, Head of Cloud Operations, Kaplan

Application communication is evolving
Functional calls
Find endpoints,
then connect
Across the room
Dynamic name,
number, and location
Across cities 1:1
Known endpoints, APIs

Developers need to connect micro services
Build apps
invoking other services
by name
Ensure that service
name resolves to
correct IP/port
DEV OPS

What is Service Discovery?
“Where is Service X?”
Friendly name -> IP + port
E.g., app: {10.0.4.5:8080, 10.0.4.6:8080 }

Why is it non trivial?
Dynamic by design:
Number of containers and instances
Auto assigned IP addresses and ports
Placement, scheduling, scaling
Deployments and upgrades
Health and connectivity

Decision criteria
Service registry
Discovery mechanism
Server side or client side
Registration mechanism
Third party or application framework

E.g.,
Load Balancer
Benefits
Client is simpler
Drawbacks
Install, manage
Availability, capacity
More hops
Discovery on the server side
Client
Server
Server
Server
Request Request
LB +
Service
Registry
Register
Register

Discovery on the client side
E.g.,
Route 53 Based
Benefits
Fewer hops
Drawbacks
Client must be registry aware
Client implements discovery logic
Client
Server
Server
Server
Service
Registry
Request
Register
Register
Register
Query

Registration by third party
E.g.,
Consul, Etcd and Zookeeper
Benefits
Registrar registers, unregisters
and performs health checks
Service less complex
Drawbacks
State may not reflect whether service can
handle requests, only active or unavailable
Install, configure and manage other
component, unless it is a part of infrastructure
Server
Server
Server
Service
Registry
Update Start, stop,
health change
Registrar/
service manager

Registration by service language framework
E.g.,
Microservice chassis frameworks
Java: Spring Boot, Spring Cloud, Dropwizard
Go: Gizmo, Micro and Go kit
Benefits
Potentially more states: e.g., STARTING, AVAILABLE, etc.
AVAILABLE, etc.
Drawbacks
Couples the service to the Service Registry
Must implement registration logic in each programming
programming language/framework
Service/frameworks often lack the
self-awareness to unregister
Server
Server
Server
Service
Registry
Start, stop,
health change

Patterns for Service Discovery
Load Balancers Service MeshDNS Key/Value store

1. Using Load Balancers
Server side service discovery
Service Registry = Load Balancer
target groups
Registry update done by ECS
scheduler
Similar architecture with Elastic
Load Balancer, Network Load
Balancer,
or Application Load Balancer
LB

2. Using DNS, e.g., ECS + Route 53 (New!)
Cluster: myapp
App
10.0.6.5:8080
Web
10.0.8.6:8080
App
10.0.4.5:8080
Web
10.0.3.6:8080
Namespace: myapp.local
Service: web.myapp.local
10.0.4.5:8080,10.0.3.6:80
80
Service: app.myapp.local
10.0.6.5:8080,10.0.8.6:80
80
Client side service discovery
Service Registry = Route53 DNS
Server
Registry update done by ECS
scheduler
Managed, high availability,
high scale, extensible

3. Using key-value stores, e.g., Consul
Service
registry
Client side Service Discovery
Service Registry = Route53 DNS Server
Registry update done by ECS scheduler
Managed, high availability, high scale,
extensibleECS Services
Consul-agent
Registrator
Weather
Stock-price
Portal
ECS Services
Registrator
Consul-agent
ECS Instance ECS Instance
ECS Cluster
ECS Instance
Consul-agent

4. Using Service Mesh e.g., Linkerd, Envoy
Service Mesh handles service discovery
Linkerd/Envoy proxy maintain
service registry
Registry updates done by third party tool
like Consul

Microservices at Kaplan Test Prep
Roven Drabo, Head of Cloud Operations
Customer case study

Kaplan Test Prep
Offers preparation for
more than 90
standardized tests,
including entrance exams
for secondary school,
college, and graduate
school
Professional
Licensing
exams
Provides private
tutoring and graduate
admissions consulting

Our business challenges
Reduce time to
market (TTM) for
new products and
features
Reduce IT
infrastructure
and operational
costs
Improve reliability and
scalability of our
core applications
Improve engineers
overall satisfaction
and level of
happiness

Architecture and engineering challenges
Large systems-to-services ratio; more systems equals more
operational overhead
Manual release process, no CI/CD pipelines, and complex scripts to
maintain
Legacy monolithic apps and web services needed to be transformed
into infrastructure as code and rearchitected along the way

Infrastructure requirements and needs
Needs
Need for an orchestration which would fit our
needs
CloudFormation to automate and deploy AWS
services
Consul Key/Value Store as a global state of the
microservices
NGINX for Routing, Customization and Proxying
Requirements
Easy Cluster Management
Container
Management/Scheduling
Auto healing/scaling
AWS Integration

SSDR high level architecture

Build and deploy in < 2 minutes
udeploy qa create –s
/mystack/myfiles.yaml
CFML
CloudFormation
template
SSDR
Platform

udeploy example.yaml

udeploy CLI deployment example

Creates CloudFormation stack and substacks

Service up and running

example.yaml with unencrypted password

Password encrypted using udeploy

example.yaml with encrypted password

Summary
Flipped the ratio of systems-to-services from 3-1 to 1-15
Code deployments now require 1 person and take on average < than 5
mins
Enabled self service for developers to push code to quality assurance,
staging and production through a CI/CD pipeline
Operational workload has diminished and improve release cycles
% of code covered by automated testing grew from 50% to 80%
Infrastructure savings for migrated apps average 80%

Thank you

Patterns and Considerations in Service Discovery - Con327 - re:Invent 2017

More Related Content

What's hot

Similar to Patterns and Considerations in Service Discovery - Con327 - re:Invent 2017

Recently uploaded

Patterns and Considerations in Service Discovery - Con327 - re:Invent 2017