Microservices are a software architecture style where applications are composed of small, independent services that communicate using language-agnostic APIs. Microservices are designed to be small, highly decoupled, and focus on doing a single task. This contrasts with monolithic architectures that use a single codebase. Microservices architectures enable independent development, deployment, and scaling of services. However, running microservices at scale introduces challenges around resource management, monitoring, service discovery, and deployment complexity.

1. Introduction to Microservices

2. What Are Microservices?
“is a software architecture style in which complex
applications are composed of small, independent
processes communicating with each other using language-
agnostic APIs. These services are small, highly
decoupled and focus on doing a small task, facilitating a
modular approach to system-building.” - Wikipedia
https://en.wikipedia.org/wiki/Microservices

3. Monolithic vs. SOA vs. Microservices
SOA
Coarse-
grained
Microservices
Fine-grained
Monolithic
Single Unit

4. Order UI User UI
Shipping
UI
Order
Service
User
Service
Shipping
Service
Data
Access
Monolithic Architecture

5. Monolithic Architecture – Scaling

6. developers
releasetestbuild
delivery pipelineapp
Monolithic Development Lifecycle

7. Order UI User UI
Shipping
UI
Order
Service
User
Service
Shipping
Service
Microservices Architecture

8. Order UI User UI UI
Order
Service
Service
Shipping
Service
Order UI
Order UI
User UI UIShipping
UI
Order
ServiceOrder
Service
Service
Service
Service
Service
User
Service
Shipping
Service
Microservices Architecture – Scaling

9. Microservices Development Lifecycle
developers delivery pipelinemicroservices
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline
build pipeline

10. Do one
thing well
Independent
Decentralized
Black box
Polyglot
You build it, you run it
Characteristics of Microservices Architectures

11. Challenges in running
Microservices

12. Microservice Challenge #1 – Resource Management
Managing a large fleet by hand is impossible:
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
AZ 1 AZ 2
AZ 3

13. Microservices Challenge #2 – Monitoring
A microservices architecture will have 10s, 100s, 1000s,
maybe even 10,000s of individual services:
• How do you know if an individual service is healthy?
• How do you measure the performance of an individual
service?
• How do you troubleshoot and debug an individual
service?

14. Microservices Challenge #3: Service Discovery
Each microservice scales up and down independently of
one another:
• How does Service A know the URLs for all instances of
Service B?
• How do you allow services to scale independently while
still using load balancers?
• How does a new instance of a service announce itself to
other services?

15. Microservices Challenge #4: Deployment
A microservices architecture will have 10s, 100s, 1000s,
maybe even 10,000s of individual services:
• Each service will be developed, tested, and deployed on
its own timeline – How do you manage this across large
numbers of services?
• Services are polyglot – different languages, frameworks
– how do you efficiently deploy them?
• How do you decide which hosts to deploy a service on?

16. Amazon EC2 Container Service

17. Containers Are Natural for Services
Simple to model
Any app, any language
Image is the version
Test & deploy same artifact
Stateless servers decrease change risk

18. Easily Manage Clusters for Any Scale
Nothing to run
Complete state
Control and monitoring
Scale

19. Designed for Use with Other AWS Services
Elastic Load Balancing
Amazon Elastic Block Store
Amazon Virtual Private Cloud
AWS Identity and Access Management
AWS CloudTrail

20. Clusters
Regional
Resource pool
Grouping of container instances
Start empty, dynamically scalable

21. Tasks
Unit of work
Grouping of related containers
Run on container instances

22. Services
Good for long-running applications
Load balance traffic across containers
Automatically recover unhealthy containers
Discover services

23. Monitoring with Amazon CloudWatch
Metric data sent to CloudWatch in 1-minute periods and recorded for a period
of two weeks
Available metrics: CPUReservation, MemoryReservation, CPUUtilization,
MemoryUtilization
Available dimensions: ClusterName, ServiceName

24. Service
Discovery
with Route53
and
Application
Load
Balancing

25. Scheduling Containers on ECS
Batch Jobs
ECS task scheduler
Run tasks once
Batch jobs
RunTask (random)
StartTask (placed)
Long-Running Apps
ECS service scheduler
Health management
Scale-up and scale-down
AZ aware
Grouped containers

26. Scheduling Containers: Long-Running App
Deploy using the least space: minimumHealthyPercent =
50%, maximumPercent = 100%

27. Scheduling Containers: Long-Running App
Deploy quickly without reducing service capacity:
minimumHealthyPercent = 100%, maximumPercent =
200%

28. Scheduling Containers: Long-Running App
Blue-Green Deployments
• Define two ECS services
• Each service is associated w/ ELB
• Both ELBs in Route 53 record set
with weighted routing policy, 100%
primary, 0% secondary
• Deploy to blue or green service and
switch weights
TaskTask
Route 53
record set
with
weighted
routing
policy
0%
100%

29. Introduction to Serverless

30. Servers
Which users should have
access to my servers?
How will the application
handle server hardware failure?
How many users create
too much load for my servers?
How can I control
access from my servers?
When should I decide to
scale out my servers?
When should I decide to
scale up my servers?
How will I keep my server
OS patched?
Which OS should my
servers run?
What size servers are
right for my budget?
How can I tell if a
server has been
compromised?
Which packages should
be baked into my server images?
How will new code be
deployed to my servers?
How should I implement dynamic
configuration changes on my servers
How much remaining
capacity do my servers have?
(AAHHHHHHHHH!!)

31. Evolution of computing

32. What is serverless computing?
EC2
• Machine as the
unit of scale
• Abstracts the
hardware
• I configure
machines,
storage,
networking, OS
ECS
• Application as
the unit of scale
• Abstracts the
OS
• I run servers,
configure apps,
control scaling
AWS Lambda
• Functions as
the unit of scale
• Abstracts the
language
runtime
• I run my code
when it’s
needed

33. What is serverless computing?
EC2 ECS AWS Lambda

34. The serverless compute manifesto
Functions are the unit of deployment and scaling.
No machines, VMs, or containers visible in the programming model.
Permanent storage lives elsewhere.
Scales per request. Users cannot over- or under-provision capacity.
Never pay for idle (no cold servers/containers or their costs).
Implicitly fault-tolerant because functions can run anywhere.
BYOC – Bring your own code.
Metrics and logging are a universal right.

35. What is AWS Lambda?

36. What is AWS Lambda?
EVENT SOURCE FUNCTION SERVICES (ANYTHING)
Changes in
data state
Requests to
endpoints
Changes in
resource
state
Node.js
Python
Java
C# NEW!
Triggers Uses

37. Continuous scalingNo servers to
manage
Never pay for idle
– No cold servers
(only happy
accountants)
Benefits of AWS Lambda

38. When to use Lambda?

39. Data stored in
Amazon
DynamoDB
Dynamic content in
AWS Lambda
Amazon API
Gateway
Browser
Amazon
CloudFront
Amazon S3
Web Application use case

40. Reference Case: Alt/S Sportify – sportify.io

41. Using AWS Lambda
Bring your own code
• Node.js, Java, Python, C#
• Bring your own libraries
(even native ones)
Simple resource model
• Select power rating from 128
MB to 1.5 GB
• CPU and network allocated
proportionately
• Reports actual usage
Flexible authorization
• Securely grant access to
resources, including VPCs
• Fine-grained control over
who can call your functions
Flexible use
• Call or send events
• Integrated with other AWS
services
• Build whole serverless
ecosystems

42. Using AWS Lambda
Programming model
• AWS SDK built in (Python and
Node.js)
• Eclipse & Visual Studio plugins
• AWS Lambda is the “webserver”
• Use processes, threads, /tmp,
sockets normally
Stateless
• Persist data using Amazon
DynamoDB, S3, or
ElastiCache
• No affinity to infrastructure
(can’t “log in to the box”)
Authoring functions
• Author directly using the
console WYSIWYG editor
• Package code as a .zip and
upload to AWS Lambda or
S3
• Command line tools
Monitoring and logging
• Built-in metrics for requests,
errors, latency, and throttles
• Built-in logs in Amazon
CloudWatch Logs

43. Stream processing use case
• High ingest rate
• Near real-time processing (low latency from ingest to
process)
• Spiky traffic (lots of devices with intermittent network
connections)
• Message durability
• Message ordering

44. Serverless stream processing architecture
Sensors
Amazon Kinesis:
Stream
Lambda:
Stream Processor
S3:
Final Aggregated Output
Lambda:
Periodic Dump to S3
CloudWatch Events:
Trigger every 5 minutes
S3:
Intermediate Aggregated
Data
Lambda:
Scheduled Dispatcher
KPL:
Producer

45. Reference Case: bustle.com
Amazon
API Gateway
AWS Lambda Amazon Kinesis AWS Lambda
Redis
Amazon Mobile
Analytics
Amazon CloudWatch
Amazon Elasticsearch
Service
Amazon S3
Amazon RedshiftAmazon QuickSight
Engineering
Marketing & Operations
Design
Bustle.com users
• 52 million monthly users
• 100 million events daily
• 84% cost savings
• 0 servers
• 0 operating system patches
• Automatic scaling

46. Using Serverless to Extend the AWS
Platform

47. Extending AWS Services using AWS Lambda
AWS
CloudFormation
custom
resources
Amazon Echo skills Amazon SWF tasks Customized
notifications with
Amazon SNS
Amazon Cognito
triggers
Amazon S3
triggers
Amazon
Dynamo DB
triggers
Amazon
Kinesis
processors
Microservices
with API
Gateway
Alexa, do
my expense
report
And the list
continues
to grow!

48. Reference Case: AU Financial Enterprise Customer
DevOps CloudFormation AWS
Config
AWS
Lambda
EC2
RDS
IAM Role
S3 Bucket
Deploy
Creates
Audit
Tagging
Compliance
AWS
Lambda
AWS
Lambda
AWS
Lambda
Audit
Encryption
Compliance
Audit HA,
Encryption,
Backup
Compliance
Audit
Services
Compliance
Triggers