Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Microservice Architecture on AWS
using AWS Lambda
and Docker Containers
Danilo Poccia ‒ AWS Technical Evangelist
@danilop
Why Microservices?
http://en.wikipedia.org/wiki/Multitier_architecture
As a Project scales Complexity arises
“Complexity arises when the
dependencies among the elements
become important.”
Compl...
How to design

smaller services?
Business Domain
+
Loosely Coupled
+
Bounded Context
Microservices
Independent Deployment
Microservices
Choose the Right Tool
Microservices
Adopt New Technologies
Microservices
Culture of Automation
Adrian Cockcroft, Technology Fellow at Battery Ventures
http://www.slideshare.net/adriancockcroft/goto-berlin
How small is small?
“something that could be
rewritten in two weeks”
Two Pizza Teams
Conway’s law
“organizations which design systems ...
are constrained to produce designs which
are copies of the communicat...
Architecture
DevOps
M
icroservices
Distributed Systems
Independent Scalability
Auto Scaling
Security
Least Privileges
Single Sign-On
Confused Deputy Problem
(for downstream calls
after authentication)
Testing
Automate
Service Tests
End-to-end Tests
Synthetic Monitoring
Monitoring
Expose Service Metrics
Standard Log Format
Correlation ID
From Log collection to
Real-Time Event Routing
Amazon...
Discovery
Self-Describing System
Amazon Route 53
(DNS)
AWS Resource
Tagging
Deployment Pipeline
Developer
Developer
Developer
Developer
Developer
Build
Stage #1 Stage #2 … Stage #N Production
Build Build Build Build
Developer
Developer
Developer
Developer
Developer
Build
Stage #1 Stage #2 … Stage #N Production
Build Build Build Build
Bu...
Developer
Developer
Developer
Developer
Developer
Build
Stage #1 Stage #2 … Stage #N Production
Build Build Build Build
Bu...
Developer
Developer
Developer
Developer
Developer
Build
Stage #1 Stage #2 … Stage #N Production
Build Build Build Build
Bu...
Design for Failure
Degrade Functionality
"Jtecul" by own - Own work.
Licensed under CC BY-SA 3.0 via Wikimedia Commons
http://commons.wikimedia.org/wiki/File:Jtecu...
Elastic
Load Balancing
Health
Checks
Service
Consumer(s)
Consumer(s)
Amazon Route 53
Private DNS in VPC
Health
Checks
Service
Consumer(s)
Consumer(s)
Amazon Route 53
Private DNS in VPC
Elastic
Load Balancing
Health
Checks
Service
Consumer(s)
Consumer(s)
Disposable Infrastructure
Immutable Deployments
Why Docker?
Virtual Machine Container
Docker on AWS
Amazon
Linux
A supported and
maintained Linux
image provided by
Amazon Web Services
Amazon EC2
Container
Ser...
Amazon EC2 Container Service
Key Components
Container Instances
Clusters
Tasks
Task Definitions
Amazon EC2 instances
Docker...
https://github.com/aws/amazon-ecs-agent
Amazon EC2 Container Service
Key Components
Container Instances
Clusters
Tasks
Task Definitions
Regional
Resource pool
Grou...
Amazon EC2 Container Service
Key Components
Container Instances
Clusters
Tasks
Task Definitions
Unit of work
Grouping of re...
Amazon EC2 Container Service
Key Components
Container Instances
Clusters
Tasks
Task Definitions
[
{
"image": "mysql",
"name...
Tasks are defined via Task Definitions
[
{
"image": "tutum/wordpress-stackable",
"name": "wordpress",
"cpu": 10,
"memory":...
[
{
"image": "tutum/wordpress-stackable",
"name": "wordpress",
"cpu": 10,
"memory": 500,
"essential": true,
"links": [
"db...
[
{
"image": "tutum/wordpress-stackable",
"name": "wordpress",
"cpu": 10,
"memory": 500,
"essential": true,
"links": [
"db...
Why AWS Lambda?
AWS Lambda
Event driven,
fully managed compute
AWS Lambda
Focus on business logic,
not infrastructure
Customer uploads code,
AWS Lambda handles:
Capacity
Scaling
Deploym...
AWS Lambda
Automatic scaling
Customers pay only
for what they use,
no over/under provisioning
AWS Lambda
Fine-grained pricing
Price compute time by 100ms,
even short jobs make sense
Low request charge
No hourly, dail...
Events come in many different shapes & sizes
S3 event
notifications
DynamoDB
Streams
Kinesis events Custom
events
Amazon EC2 Containers AWS LambdaOn-Premises
Weeks Minutes Seconds Milliseconds
Netflix Architecture
Asgard, Amimator, Hystrix,
Cassandra, JVM, Docker, …
O
n
AW
S
Hailo Architecture
Hubot, Janky, Jenkins,
Go, RabbitMQ, Cassandra,
Docker, …
O
n
AW
S
Demo Architecture
Content Management System
Prototype
O
n
AW
S
Lambda
Function
S3 Bucket

#1
S3 Bucket

#2
DynamoDB

Table
ECS

Task
ECS

Cluster
Images
+

Medatada
Upload Object
+ Meta...
<demo>
…
</demo>
Iterate
Continuous Improvement
Kaizen
@danilop
Microservice Architecture on AWS using AWS Lambda and Docker Containers
Microservice Architecture on AWS using AWS Lambda and Docker Containers
Upcoming SlideShare
Loading in …5
×

Microservice Architecture on AWS using AWS Lambda and Docker Containers

16,530 views

Published on

The use of microservices as an architectural pattern, decomposing an application into small, independent components, can improve development, deployment and security. We’ll build a real architecture using AWS Lambda to run event-based functions and Amazon EC2 Container Service and AWS Elastic Beanstalk to manage backend and frontend Docker containers. We’ll evolve from a web based interface to a mobile, cross platform architecture, using a least-privilege approach on security based on AWS Identity and Access Management roles.

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Microservice Architecture on AWS using AWS Lambda and Docker Containers

  1. 1. Microservice Architecture on AWS using AWS Lambda and Docker Containers Danilo Poccia ‒ AWS Technical Evangelist @danilop
  2. 2. Why Microservices?
  3. 3. http://en.wikipedia.org/wiki/Multitier_architecture
  4. 4. As a Project scales Complexity arises “Complexity arises when the dependencies among the elements become important.” Complex Adaptive Systems: An Introduction to Computational Models of Social Life Scott E. Page, John H. Miller
  5. 5. How to design
 smaller services?
  6. 6. Business Domain + Loosely Coupled + Bounded Context
  7. 7. Microservices Independent Deployment
  8. 8. Microservices Choose the Right Tool
  9. 9. Microservices Adopt New Technologies
  10. 10. Microservices Culture of Automation
  11. 11. Adrian Cockcroft, Technology Fellow at Battery Ventures http://www.slideshare.net/adriancockcroft/goto-berlin
  12. 12. How small is small?
  13. 13. “something that could be rewritten in two weeks”
  14. 14. Two Pizza Teams
  15. 15. Conway’s law “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations” Melvin Conway, 1968
  16. 16. Architecture DevOps M icroservices
  17. 17. Distributed Systems Independent Scalability Auto Scaling
  18. 18. Security Least Privileges Single Sign-On Confused Deputy Problem (for downstream calls after authentication)
  19. 19. Testing Automate Service Tests End-to-end Tests Synthetic Monitoring
  20. 20. Monitoring Expose Service Metrics Standard Log Format Correlation ID From Log collection to Real-Time Event Routing Amazon CloudWatch Logs Amazon Kinesis Understand your Trends
  21. 21. Discovery Self-Describing System Amazon Route 53 (DNS) AWS Resource Tagging
  22. 22. Deployment Pipeline
  23. 23. Developer Developer Developer Developer Developer Build Stage #1 Stage #2 … Stage #N Production Build Build Build Build
  24. 24. Developer Developer Developer Developer Developer Build Stage #1 Stage #2 … Stage #N Production Build Build Build Build Bug!
  25. 25. Developer Developer Developer Developer Developer Build Stage #1 Stage #2 … Stage #N Production Build Build Build Build Build Build Build Build Build Build Build Build Build Build
  26. 26. Developer Developer Developer Developer Developer Build Stage #1 Stage #2 … Stage #N Production Build Build Build Build Bug! Build Build Build Build Build Build Build Build Build Build
  27. 27. Design for Failure Degrade Functionality
  28. 28. "Jtecul" by own - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons http://commons.wikimedia.org/wiki/File:Jtecul.jpg#/media/File:Jtecul.jpg Circuit Breakers
  29. 29. Elastic Load Balancing Health Checks Service Consumer(s) Consumer(s)
  30. 30. Amazon Route 53 Private DNS in VPC Health Checks Service Consumer(s) Consumer(s)
  31. 31. Amazon Route 53 Private DNS in VPC Elastic Load Balancing Health Checks Service Consumer(s) Consumer(s)
  32. 32. Disposable Infrastructure Immutable Deployments
  33. 33. Why Docker?
  34. 34. Virtual Machine Container
  35. 35. Docker on AWS Amazon Linux A supported and maintained Linux image provided by Amazon Web Services Amazon EC2 Container Service Highly scalable, high performance container management service AWS Elastic Beanstalk For deploying and scaling web applications and services
  36. 36. Amazon EC2 Container Service Key Components Container Instances Clusters Tasks Task Definitions Amazon EC2 instances Docker daemon Amazon ECS agent
  37. 37. https://github.com/aws/amazon-ecs-agent
  38. 38. Amazon EC2 Container Service Key Components Container Instances Clusters Tasks Task Definitions Regional Resource pool Grouping of Container Instances Start empty, dynamically scalable
  39. 39. Amazon EC2 Container Service Key Components Container Instances Clusters Tasks Task Definitions Unit of work Grouping of related Containers Run on Container Instances
  40. 40. Amazon EC2 Container Service Key Components Container Instances Clusters Tasks Task Definitions [ { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, …
  41. 41. Tasks are defined via Task Definitions [ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ … ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] } ]
  42. 42. [ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ … ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, ] { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] } ] From Docker Hub 10 CPU Units (1024 is full CPU), 500 Megabytes of Memory Environment Variables No external ports exposed Tasks are defined via Task Definitions
  43. 43. [ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ … ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, [ { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] } ] Essential to our Task Docker link to mysql container Expose port 80 in container to port 80 on host Tasks are defined via Task Definitions
  44. 44. Why AWS Lambda?
  45. 45. AWS Lambda Event driven, fully managed compute
  46. 46. AWS Lambda Focus on business logic, not infrastructure Customer uploads code, AWS Lambda handles: Capacity Scaling Deployment Fault tolerance Monitoring Logging . . .
  47. 47. AWS Lambda Automatic scaling Customers pay only for what they use, no over/under provisioning
  48. 48. AWS Lambda Fine-grained pricing Price compute time by 100ms, even short jobs make sense Low request charge No hourly, daily, or monthly minimums Free tier
  49. 49. Events come in many different shapes & sizes S3 event notifications DynamoDB Streams Kinesis events Custom events
  50. 50. Amazon EC2 Containers AWS LambdaOn-Premises Weeks Minutes Seconds Milliseconds
  51. 51. Netflix Architecture Asgard, Amimator, Hystrix, Cassandra, JVM, Docker, … O n AW S
  52. 52. Hailo Architecture Hubot, Janky, Jenkins, Go, RabbitMQ, Cassandra, Docker, … O n AW S
  53. 53. Demo Architecture Content Management System Prototype O n AW S
  54. 54. Lambda Function S3 Bucket #1 S3 Bucket #2 DynamoDB
 Table ECS
 Task ECS
 Cluster Images + Medatada Upload Object + Metadata Event Notification Thumbnail Write Metadata Run Schedule Read Metadata Write “index.html” 1 2 3
  55. 55. <demo> … </demo>
  56. 56. Iterate Continuous Improvement Kaizen
  57. 57. @danilop

×