Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Adrian Trenaman
SVP Engineering, gilt.com
@adria...
Gilt’s Journey
Gilt: Luxury designer brands at members-only prices
... we shoot the product in our studios
... we receive, store, pack, and ship ...
... we sell every day at noon EST
… stampede!
... this is what noon really looks like.
How It All Started…
From Rails to Riches ― 2007: A Ruby on Rails monolith
Jobs
Ruby on Rails memcache
Postgres
2011: Java, loosely-typed, monolithic services
(1) Large,
loosely typed
services
(2) Teams
focused on
business lines
(3) M...
“How can we arrange our teams around
strategic initiatives? How can we make it fast
and easy to get to change to productio...
2015: LOSA (Lots of Small Apps) & Microservices
Driving Forces Behind Gilt’s Emergent
Architecture
• Team autonomy
• Voluntary adoption (tools, techniques, processes)
• K...
Service growth over time: point of inflexion === Scala.
What are all these services doing?
Anatomy of a service
Anatomy of a gilt service – typical choices
gilt-service-
framework,
, Java, Javascript
Log4j, Cloudwatch
Cave
or
Lines of code per service (logarithmic scale)
# source files per service (includes build, config, xml, Java, Scala, Ruby...)
Service discovery: straightforward
ZooKeeper
Brocade Traffic Manager (aka Zeus,
Stringray, SteelApp,...)
From bare metal…
PHX
IAD
… to vapour.
Lift-and-shift + elastic teams
Existing Data Centre
Dual 10-Gb direct connect line, 2-ms latency.
“Legacy VPC”
MobileCommo...
Single-tenant deployment: one service per EC2 instance
Reproducible, immutable deployments: Docker
Service discovery: new services use ELB
ZooKeeper
Elastic Load
Balancing (ELB)
# running instances per service: “rule of three” (previously “rule of four”)
EC2 instance sizing: lots of small instances
Evolution of architecture and tech organization
We (heart) μ-services
• Lessen dependencies between
teams: faster code-to-prod
• Lots of initiatives in parallel
• Your fa...
We (heart) cloud
• Do DevOps in a meaningful
way.
• Low barrier of entry for new
tech (Amazon DynamoDB,
Amazon Kinesis,......
Common Challenges and
Patterns
Monolithic Microservices
• Simple deployments
• Binary failure modes
• Inter-module refactoring
• Technology monoculture
•...
• Organization
• Discovery
• Data management
• Deployment
• I/O explosion
• Monitoring
Common Challenges and Patterns
Organization
Monolithic Ownership
Organized on technology capabilities
UI Team
DBA Team
App Logic Team
Web Tier
App Tier
DB
Organizatio...
Microservices Ownership
Organized on business responsibilities
Login
Registration
Order
Personalization
Accounts team
Mobi...
Microservices Ownership
• Requirements
• Technology selection
• Development
• Quality
• Deployment
• Support
How to Be a Good Citizen (Service Consumer)
• Design for failure
• Expect to be throttled
• Retry w/ exponential backoff
•...
How to Be a Good Citizen (Service Provider)
• Publish your metrics
• Protect yourself
• Keep your implementation details p...
Amazon API Gateway
• Throttling (global and per-method)
• Caching (with TTLs and invalidation)
• Monitoring (RPS, latency,...
Discovery
Use DNS
Convention-based naming
<service-name>-<environment>.domain.com
shoppingcart-gamma.example.com
<service-name>.<env...
Use a Dynamic Service Registry
• Avoids the DNS TTL issue
• More than service registry & discovery
• Configuration managem...
Data management
Challenge: Centralized Database
Monolithic applications typically
have a monolithic data store:
• Difficult to make schema...
Centralized Database – Anti-pattern
Monolithic applications typically
have a monolithic data store:
• Difficult to make sc...
Decentralized Data Stores
• Each service chooses its data
store technologies
• Low impact schema changes
• Independent sca...
Challenge: Transactional Integrity
• Use a pessimistic model
• Handle it in the client
• Add a transaction manager / distr...
Challenge: Aggregation
• Pull: Make the data available via your service API
• Push: To Amazon S3, Amazon CloudWatch, or an...
Deployment
Continuous Delivery & Continuous Deployment
Create the right build pipeline for each service
• AWS CodeDeploy
• AWS Elasti...
AWS CodePipeline
Multiple Services per Container/Instance
• Independent monitoring
• Independent scaling
• Clear ownership
• Immutable depl...
Multiple Services per Container/Instance – Anti-pattern
• Independent monitoring
• Independent scaling
• Clear ownership
•...
Single Service per Container/Instance
• Independent monitoring
• Independent scaling
• Clear ownership
• Immutable deploym...
…Or Just Use AWS Lambda
LambdaAPI Gateway
RDS
DynamoDB
I/O explosion
Challenge: Request Multiplication
checkout-
svc
cart-svc
account-svc
user-svc
Single request
Add Client Caching
checkout-
svc
cart-svc
account-svc
user-svc
Single request
cache
user & account cache
Challenge: Hotspots
cart-svc order-svc
shipping-
svc
user-svc
single request
get (user x, col y) get (user x, col y) get (...
Use Dependency Injection
cart-svc order-svc
shipping-
svc
user-svc
single request
get (user x, col y)
(user x, col y) (use...
Monitoring
Challenge: monitoring
• Publish externally relevant metrics
• Latency
• RPS
• Error rate
• Understand internally relevant ...
Challenge: Logging
• Pick a common log aggregation solution
• Agree on log entry formats
• Use naming conventions
• Agree ...
Challenge: Correlating Requests
ui-svc catalog-
svc
checkout-
svc
shipping-
svc
payment-
svc
request
Use Correlation IDs
09-02-2015 15:03:24 ui-svc INFO [uuid-123] ……
09-02-2015 15:03:25 catalog-svc INFO [uuid-123] ……
09-02...
What did we cover?
• Ownership
• Discovery
• Data management
• Deployment
• I/O explosion
• Monitoring
Related Sessions
• ARC201 - Microservices Architecture for Digital Platforms with AWS
Lambda, Amazon CloudFront and Amazon...
Remember to complete
your evaluations!
Thank you!
Adrian Trenaman
SVP Engineering, gilt.com
@adrian_trenaman
Derek Chiles
Sr. Mgr, Solutions Architecture, AWS
@d...
Upcoming SlideShare
Loading in …5
×

(ARC309) Getting to Microservices: Cloud Architecture Patterns

8,747 views

Published on

Gilt, a billion dollar e-commerce company, implemented a sophisticated microservices architecture on AWS to handle millions of customers visiting their site at noon every day. The microservices architecture pattern enables independent service scaling, faster deployments, better fault isolation, and graceful degradation. In this session, Derek Chiles, AWS solutions architect, will review best practices and recommended architectures for deploying microservices on AWS. Adrian Trenaman, SVP of engineering at Gilt, will share Gilt's experiences and lessons learned during their evolution from a single monolithic Rails application in a traditional data center to more than 300 Scala/Java microservices deployed in the cloud.

Published in: Technology
  • Be the first to comment

(ARC309) Getting to Microservices: Cloud Architecture Patterns

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Adrian Trenaman SVP Engineering, gilt.com @adrian_trenaman October 2015 From Monolithic to Microservices Evolving Architecture Patterns in the Cloud Derek Chiles Sr. Mgr, Solutions Architecture, AWS @derekchiles ARC309
  2. 2. Gilt’s Journey
  3. 3. Gilt: Luxury designer brands at members-only prices
  4. 4. ... we shoot the product in our studios
  5. 5. ... we receive, store, pack, and ship ...
  6. 6. ... we sell every day at noon EST
  7. 7. … stampede!
  8. 8. ... this is what noon really looks like.
  9. 9. How It All Started…
  10. 10. From Rails to Riches ― 2007: A Ruby on Rails monolith Jobs Ruby on Rails memcache Postgres
  11. 11. 2011: Java, loosely-typed, monolithic services (1) Large, loosely typed services (2) Teams focused on business lines (3) Monolithic Java application; huge bottleneck for innovation (4) Hidden linkages; buried business logic
  12. 12. “How can we arrange our teams around strategic initiatives? How can we make it fast and easy to get to change to production?” Enter: µ-services
  13. 13. 2015: LOSA (Lots of Small Apps) & Microservices
  14. 14. Driving Forces Behind Gilt’s Emergent Architecture • Team autonomy • Voluntary adoption (tools, techniques, processes) • KPI / goal-driven initiatives • Failing fast and openly • Being open and honest, even when it’s difficult
  15. 15. Service growth over time: point of inflexion === Scala.
  16. 16. What are all these services doing?
  17. 17. Anatomy of a service
  18. 18. Anatomy of a gilt service – typical choices gilt-service- framework, , Java, Javascript Log4j, Cloudwatch Cave or
  19. 19. Lines of code per service (logarithmic scale)
  20. 20. # source files per service (includes build, config, xml, Java, Scala, Ruby...)
  21. 21. Service discovery: straightforward ZooKeeper Brocade Traffic Manager (aka Zeus, Stringray, SteelApp,...)
  22. 22. From bare metal… PHX IAD
  23. 23. … to vapour.
  24. 24. Lift-and-shift + elastic teams Existing Data Centre Dual 10-Gb direct connect line, 2-ms latency. “Legacy VPC” MobileCommon Person- alisation Admin Data (1) Deploy to VPC (2) “Department” accounts for elasticity & DevOps
  25. 25. Single-tenant deployment: one service per EC2 instance
  26. 26. Reproducible, immutable deployments: Docker
  27. 27. Service discovery: new services use ELB ZooKeeper Elastic Load Balancing (ELB)
  28. 28. # running instances per service: “rule of three” (previously “rule of four”)
  29. 29. EC2 instance sizing: lots of small instances
  30. 30. Evolution of architecture and tech organization
  31. 31. We (heart) μ-services • Lessen dependencies between teams: faster code-to-prod • Lots of initiatives in parallel • Your favourite <tech/language/framework> here • Graceful degradation of service • Disposable code: easy to innovate, easy to fail and move on.
  32. 32. We (heart) cloud • Do DevOps in a meaningful way. • Low barrier of entry for new tech (Amazon DynamoDB, Amazon Kinesis,...) • Isolation • Cost visibility • Security tools (IAM) • Well documented • Resilience is easy • Hybrid is easy • Performance is great
  33. 33. Common Challenges and Patterns
  34. 34. Monolithic Microservices • Simple deployments • Binary failure modes • Inter-module refactoring • Technology monoculture • Vertical scaling • Partial deployments • Graceful degradation • Strong module boundaries • Technology diversity • Horizontal scaling
  35. 35. • Organization • Discovery • Data management • Deployment • I/O explosion • Monitoring Common Challenges and Patterns
  36. 36. Organization
  37. 37. Monolithic Ownership Organized on technology capabilities UI Team DBA Team App Logic Team Web Tier App Tier DB Organizational Structure Application Architecture
  38. 38. Microservices Ownership Organized on business responsibilities Login Registration Order Personalization Accounts team Mobile Personalization team Mobile team
  39. 39. Microservices Ownership • Requirements • Technology selection • Development • Quality • Deployment • Support
  40. 40. How to Be a Good Citizen (Service Consumer) • Design for failure • Expect to be throttled • Retry w/ exponential backoff • Degrade gracefully • Cache when appropriate
  41. 41. How to Be a Good Citizen (Service Provider) • Publish your metrics • Protect yourself • Keep your implementation details private • Maintain backwards compatibility
  42. 42. Amazon API Gateway • Throttling (global and per-method) • Caching (with TTLs and invalidation) • Monitoring (RPS, latency, error rate) • Versioning • Authentication
  43. 43. Discovery
  44. 44. Use DNS Convention-based naming <service-name>-<environment>.domain.com shoppingcart-gamma.example.com <service-name>.<environment>.domain.com shoppingcart.gamma.example.com
  45. 45. Use a Dynamic Service Registry • Avoids the DNS TTL issue • More than service registry & discovery • Configuration management • Health checks • Plenty of options • ZooKeeper (Apache) • Eureka (Netflix) • Consul (HashiCorp) • SmartStack (Airbnb)
  46. 46. Data management
  47. 47. Challenge: Centralized Database Monolithic applications typically have a monolithic data store: • Difficult to make schema changes • Technology lock-in • Vertical scaling • Single point of failure user-svc account-svccart-svc DB
  48. 48. Centralized Database – Anti-pattern Monolithic applications typically have a monolithic data store: • Difficult to make schema changes • Technology lock-in • Vertical scaling • Single point of failure user-svc account-svccart-svc DB
  49. 49. Decentralized Data Stores • Each service chooses its data store technologies • Low impact schema changes • Independent scalability • Data is gated through the service API account- svc cart- svc DynamoDB RDS user- svc ElastiCache RDS
  50. 50. Challenge: Transactional Integrity • Use a pessimistic model • Handle it in the client • Add a transaction manager / distributed locking service • Rethink your design • Use an optimistic model • Accept eventual consistency • Retry (if idempotent) • Fix it later • Write it off
  51. 51. Challenge: Aggregation • Pull: Make the data available via your service API • Push: To Amazon S3, Amazon CloudWatch, or another service you create • Pub/sub: Via Amazon Kinesis or Amazon SQS
  52. 52. Deployment
  53. 53. Continuous Delivery & Continuous Deployment Create the right build pipeline for each service • AWS CodeDeploy • AWS Elastic Beanstalk • Jenkins, CircleCI, Travis,… Integration & perf tests Build & unit tests beta Produser-svc Integration & perf tests Build & unit tests beta gamma Prodcart-svc
  54. 54. AWS CodePipeline
  55. 55. Multiple Services per Container/Instance • Independent monitoring • Independent scaling • Clear ownership • Immutable deployments user-svc cart-svc account-svc Container or instance
  56. 56. Multiple Services per Container/Instance – Anti-pattern • Independent monitoring • Independent scaling • Clear ownership • Immutable deployments user-svc cart-svc account-svc Container or instance
  57. 57. Single Service per Container/Instance • Independent monitoring • Independent scaling • Clear ownership • Immutable deployments user-svc container or instance account-svc cart-svc container or instance container or instance
  58. 58. …Or Just Use AWS Lambda LambdaAPI Gateway RDS DynamoDB
  59. 59. I/O explosion
  60. 60. Challenge: Request Multiplication checkout- svc cart-svc account-svc user-svc Single request
  61. 61. Add Client Caching checkout- svc cart-svc account-svc user-svc Single request cache user & account cache
  62. 62. Challenge: Hotspots cart-svc order-svc shipping- svc user-svc single request get (user x, col y) get (user x, col y) get (user x, col y)
  63. 63. Use Dependency Injection cart-svc order-svc shipping- svc user-svc single request get (user x, col y) (user x, col y) (user x, col y)
  64. 64. Monitoring
  65. 65. Challenge: monitoring • Publish externally relevant metrics • Latency • RPS • Error rate • Understand internally relevant metrics • Basic – CloudWatch • OS • Application
  66. 66. Challenge: Logging • Pick a common log aggregation solution • Agree on log entry formats • Use naming conventions • Agree on correlation strategy
  67. 67. Challenge: Correlating Requests ui-svc catalog- svc checkout- svc shipping- svc payment- svc request
  68. 68. Use Correlation IDs 09-02-2015 15:03:24 ui-svc INFO [uuid-123] …… 09-02-2015 15:03:25 catalog-svc INFO [uuid-123] …… 09-02-2015 15:03:26 checkout-svc ERROR [uuid-123] …… 09-02-2015 15:03:27 payment-svc INFO [uuid-123] …… 09-02-2015 15:03:27 shipping-svc INFO [uuid-123] …… ui-svc catalog- svc checkout- svc shipping- svc payment- svc request correlation id: “uuid-123” correlation id: “uuid-123”
  69. 69. What did we cover? • Ownership • Discovery • Data management • Deployment • I/O explosion • Monitoring
  70. 70. Related Sessions • ARC201 - Microservices Architecture for Digital Platforms with AWS Lambda, Amazon CloudFront and Amazon DynamoDB • CMP302 - Amazon EC2 Container Service: Distributed Applications at Scale • DEV203 - Using Amazon API Gateway with AWS Lambda to Build Secure and Scalable APIs • DVO401 - Deep Dive into Blue/Green Deployments on AWS • SPOT304 - Faster, Cheaper, Safer Products with AWS: Adrian Cockcroft Shares Experiences Helping Customers Move to the Cloud
  71. 71. Remember to complete your evaluations!
  72. 72. Thank you! Adrian Trenaman SVP Engineering, gilt.com @adrian_trenaman Derek Chiles Sr. Mgr, Solutions Architecture, AWS @derekchiles

×