AWS Scaling Microservices Architecture

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Shaun Ray
Avoid Common Mistakes
at scale
@shaun_r_ray
Nov 2018
How to:

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
My Startup Experience
https://www.amazon.com/Sun-Performance-Tuning-Java-Internet/dp/0130952494
https://www.flickr.com/photos/r4vi/191766987

Around 2010…....
0
2
4
6
8
10
12
14
16
Apr-10 Apr-10 Apr-10 May-10 May-10 May-10 May-10 May-10 Jun-10 Jun-10 Jun-10 Jun-10 Jul-10
Duration(Seconds)
Date
Average Duration per Day (2000 Transactions per Batch)

What do these pictures have in common?
Michael Jackson
Sydney Siege
Steve Irwin

Bit of an island
https://www.google.com.sg/search?q=google+maps+australia&oq=google+maps+australia

How the top news sites were doing

Don’t clear the cache

The internet is down
https://www.pinterest.com/sitecheckerpro/http-response-code/

Why I work here:
• 650 Million People
• 7 Unicorns
• Some of my customers have 90m customers
• One customer scaled to 5m users in 5 months.

Alpha Stage

Alpha Stage
?

Common Stacks
MongoDB Lamp Postgres Postgres Postgres
Express Apache Java Python Gorilla
Angular mySQL Spring Django Go
Node PHP Linux Linux Linux
1 2 3 4 12

80% of the internet

80% of the internet
PHP +
APACHE
PHP +
APACHE
mySQL
LINUX

Get some cheap hosting

You get 2x Tickets
https://www.amazon.com/Aquarius-Willy-Wonka-Golden-Ticket/dp/B01MYGLPY8

You get 2x Tickets

Don’t spend your time
on the tech

Problem #1
Each time you serve a user with a web
browser, it eats a file descriptor.
In Unix and related computer operating systems, a file
descriptor (FD, less frequently fildes) is an abstract indicator
(handle) used to access a file or other input/output
resource, such as a pipe or network socket. File descriptors
form part of the POSIX application programming interface.
A file descriptor is a non-negative integer, generally
represented in the C programming language as the type int
(negative values being reserved to indicate "no value" or an
error condition).
The default limit on Ubuntu 16.04 is 1024
# maximum capability of system
user@ubuntu:~$ cat /proc/sys/fs/file-max
708444
# available limit
user@ubuntu:~$ ulimit -n
1024
# To increase limit to say 200000
user@ubuntu:~$ sudo vim /etc/sysctl.conf
# add the following line to it
fs.file-max = 200000
https://github.com/Rayz0r/nginx

Beta Stage

Sam Newman

Transition from Monolith
Monolithic Application Services Microservices

Anatomy of a Micro-service

Data Store

Application/Logic
(code, libraries, etc)
Data Store

Application/Logic
(code, libraries, etc)
Data Store
Public API
POST /micro-service
GET /micro-service

Avoid Software Coupling

Risk Manager
micro-service
Payments
micro-service Exchange
micro-service
Orders
micro-service
Market
micro-service
Ecosystem of microservices

“Do one thing, and do it well”
“Swiss Army” by by Jim Pennucci. No alterations other than cropping. https://www.flickr.com/photos/pennuja/5363518281/
Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)

“Tools” by Tony Walmsley: No alterations other than cropping. https://www.flickr.com/photos/twalmsley/6825340663/
Image used with permissions under Creative Commons license 2.0, Attribution Generic License (https://creativecommons.org/licenses/by/2.0/)
“Do one thing, and do it well”

Problem #2
Circuit Breaker
Cascading Failure
https://martinfowler.com/bliki/CircuitBreaker.html

Growth Stage

Security and Reliability start to matter
Hey Sally, we need
to call your micro-
service to fetch
restaurants details.
Sure Paul. Which APIs you
need to call? Once I know
better your use cases I’ll
give you permission to
register your service as a
client on our service’s
directory entry.
Micro-service A Micro-service B
public API public API

Market
Micro-service
15 TPS100 TPS5 TPS20 TPS
Before we let you call
our micro-service we
need to understand
your use case,
expected load (TPS)
and accepted latency
Rate Limiting and Backoff

Standardize Logging
2016-09-02T15:03:24+00:00 ui-svc INFO [uuid-123] ...
2016-09-02T15:03:25+00:00 catalog-svc INFO [uuid-123] ...
2016-09-02T15:03:26+00:00 checkout-svc ERROR [uuid-123] ...
2016-09-02T15:03:27+00:00 payment-svc INFO [uuid-123] ...
2016-09-02T15:03:27+00:00 shipping-svc INFO [uuid-123] ...
ui-svc catalog-svc checkout-
svc
shipping-
svc
payment-
svc
request correlation id:
“uuid-123”
correlation id:
“uuid-123” correlation id:“uuid-123”

Degrade Gracefully
checkout-svc
Downstream
Service 1
Downstream
Service 2
Downstream
Service 3
Single request
Request
Request
Request
Fast response
response
Fast response
Slow
response

Cache Locally
Remote
Service
Application
Runtime
Local
Cache

It’s a journey…
Expect challenges along the way…
• Understanding of business domains
• Eventual Consistency
• Service discovery
• Lots of moving parts requires
increased coordination
• Complexity of testing / deploying /
operating a distributed system
• Cultural transformation

“Being woken up at 3am every night
by your pager is certainly a powerful
incentive to focus on quality when
writing your code.”
- Martin Fowler

AMAZON CONFIDENTIAL
Did We Scan Your Badge?
Remember to opt-in to AWS communications
and you will receive a post-event email with a
link to:
• AWS Developer Workshop Slides
• $200 in AWS Credits

AWS Scaling Microservices Architecture

Recommended

Recommended

More Related Content

Similar to AWS Scaling Microservices Architecture

Similar to AWS Scaling Microservices Architecture (20)

More from Amazon Web Services

More from Amazon Web Services (20)

AWS Scaling Microservices Architecture