Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Service Stampede
Agenda Monoliths to Microservices
Problems with microservices
Solves & Practices
The need for standardization
Introducing ...
Monolith to Microservices
Requests
Congrats! Your monolith became a thousand microservices – now you’re in serious trouble...
Cost/Benefits of Moving to Microservices
• Independence – faster PDLC
• Freedom of choice for service implementation
• Eas...
Microservices
Issues
Latency & Determinism
Service Boundaries
To be, or not to be a service
Scaling and rightsizing
Many f...
Microservices
Issues
Latency & Determinism
Service Boundaries
To be, or not to be a service
Scaling and rightsizing
Many f...
Latency Determinism
Latency by Deployment Topology
• Avoid too many layers of services
• Keep state close to the edge
• The more hops, the hig...
Microservices
Issues
Latency & Determinism
Service Boundaries
To be, or not to be a service
Scaling and rightsizing
Many f...
Microservices
Issues
Latency & Determinism
Service Boundaries
To be, or not to be a service
Scaling and rightsizing
Many f...
Services Need to Scale
• Scale horizontally with increasing workload
• More nodes, or…
• More pods with increasing workloa...
Microservices
Issues
Latency & Determinism
Service Boundaries
To be, or not to be a service
Scaling and rightsizing
Many f...
Microservices
Issues
Latency & Determinism
Service Boundaries
To be, or not to be a service
Scaling and rightsizing
Many f...
Practices for
Successful
Microservices
Deployment Topologies
Reactive Systems
Resilience with Circuit Breakers
Asynchronou...
Practices for
Successful
Microservices
Deployment Topologies
Reactive Systems
Resilience with Circuit Breakers
Asynchronou...
Individual Service Deployments
Service A Service B
RequestsRequests
Joint Deployments
Service A
Requests
Service B
Service C
• Deployment orchestration using Chef, etc.
• Kubernetes Pods
Practices for
Successful
Microservices
Deployment Topologies
Reactive Systems
Resilience with Circuit Breakers
Asynchronou...
The Reactive Manifesto
Responsive
Message Driven
Elastic Resilient
Why Does it Matter?
Respond in a deterministic, timely manner. Controls determinism
Stays responsive in the face of failur...
Practices for
Successful
Microservices
Deployment Topologies
Reactive Systems
Resilience with Circuit Breakers
Asynchronou...
Circuit Breaker Keeps systems responsive under
failure
Avoids cascading failures
Especially with multi-generational
downst...
Practices for
Successful
Microservices
Deployment Topologies
Reactive Systems
Resilience with Circuit Breakers
Asynchronou...
Practices for
Successful
Microservices
Deployment Topologies
Reactive Systems
Resilience with Circuit Breakers
Asynchronou...
Standardization
• Monitoring
• Need to collect metrics, consistently
• Logging
• Correlation across services
• Uniformity ...
Standardized Reactive Platform
Akka, Spray,
Akka Http &
Streams
Asynchronous
High Performance
Resilience & Supervision
Great Libraries for building React...
Bootstrap and
Lifecycle
Management
Unicomplex: Lightweight bootstrap
module
Emits lifecycle events: starting, active,
stop...
Listener
• Declares configuration for port binding, interfaces, security, etc
Service
• Akka Http/Spray Routes and Http Request Handler Actors
• Configured in squbs-meta.conf
• A service can be define...
Extension
• To start low level (non-actor) facilities needed for the environment
Request/Response Pipeline
Cubes
Another deployment Topology
squbs: rhymes with cubes
Drop-in modules
Cubes can run in isolation as well as on
a flat...
Orchestration
task1
task2
task3
task4
task5
Input
Output
val task1F = doTask1(input)
val task2F = doTask2(input)
val task3F = (task1F, task2F) >> doTask3
val task4F = task2F >> do...
Orchestration
DSL
High-performance asynchronous
orchestration
Responsive: Respond within SLA,
with or without results
Stre...
More Utilities
• Http Client
• Admin Console
• Actor Registry
• Perpetual Stream
• Persistence Buffer
• …
Summary
• Large number of services have benefits, but are more difficult
• Control your service topology for more determin...
Q&A – Feedback Appreciated
Service Stampede: Surviving a Thousand Services
Upcoming SlideShare
Loading in …5
×

Service Stampede: Surviving a Thousand Services

140 views

Published on

How many services do you have? 5, 10, 100? How do you even run large number of services? A micro service may be relatively simple. But services also mean distributed systems, which are inherently complex. 5 services are complex. A thousand services across many generations are at least 200 times as complex. How do we deal with such complexity?

This talk discusses service architecture at Internet scale, the need for larger transaction density, larger horizontal and vertical scale, more predictable latencies under stress, and the need for standardization and visibility. We’ll dive into how we build our latest generation service infrastructure based on Scala and Akka to serve the needs of such a large scale ecosystem.

Lastly, have the cake and eat it too. No, we’re not keeping all the goodies only to ourselves. They are all there for you in open source.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Service Stampede: Surviving a Thousand Services

  1. 1. Service Stampede
  2. 2. Agenda Monoliths to Microservices Problems with microservices Solves & Practices The need for standardization Introducing squbs
  3. 3. Monolith to Microservices Requests Congrats! Your monolith became a thousand microservices – now you’re in serious trouble!!!
  4. 4. Cost/Benefits of Moving to Microservices • Independence – faster PDLC • Freedom of choice for service implementation • Easy evolution of service & technology • Coexisting services across generations • Complexity & Latency Gains • Homogeneity • Consistency of implementation across • Timing & Determinism Losses Hmm. To be, or not to be… a service, that is...
  5. 5. Microservices Issues Latency & Determinism Service Boundaries To be, or not to be a service Scaling and rightsizing Many failure points – need resiliency Inconsistency – need standardization
  6. 6. Microservices Issues Latency & Determinism Service Boundaries To be, or not to be a service Scaling and rightsizing Many failure points – need resiliency Inconsistency – need standardization
  7. 7. Latency Determinism
  8. 8. Latency by Deployment Topology • Avoid too many layers of services • Keep state close to the edge • The more hops, the higher and less deterministic the latency is
  9. 9. Microservices Issues Latency & Determinism Service Boundaries To be, or not to be a service Scaling and rightsizing Many failure points – need resiliency Inconsistency – need standardization
  10. 10. Microservices Issues Latency & Determinism Service Boundaries To be, or not to be a service Scaling and rightsizing Many failure points – need resiliency Inconsistency – need standardization
  11. 11. Services Need to Scale • Scale horizontally with increasing workload • More nodes, or… • More pods with increasing workload • Scale vertically – why? • Keep the number of instances under control • 125 nodes @16CPU easier to manage than 1000 nodes @2CPU • Less load on network and switching infrastructure • Potentially better utilization & cache hits • Stateful systems: More limited horizontal scale • Need critical mass for redundancy
  12. 12. Microservices Issues Latency & Determinism Service Boundaries To be, or not to be a service Scaling and rightsizing Many failure points – need resiliency Inconsistency – need standardization
  13. 13. Microservices Issues Latency & Determinism Service Boundaries To be, or not to be a service Scaling and rightsizing Many failure points – need resiliency Inconsistency – need standardization
  14. 14. Practices for Successful Microservices Deployment Topologies Reactive Systems Resilience with Circuit Breakers Asynchronous Communication Standardization
  15. 15. Practices for Successful Microservices Deployment Topologies Reactive Systems Resilience with Circuit Breakers Asynchronous Communication Standardization
  16. 16. Individual Service Deployments Service A Service B RequestsRequests
  17. 17. Joint Deployments Service A Requests Service B Service C • Deployment orchestration using Chef, etc. • Kubernetes Pods
  18. 18. Practices for Successful Microservices Deployment Topologies Reactive Systems Resilience with Circuit Breakers Asynchronous Communication Standardization
  19. 19. The Reactive Manifesto Responsive Message Driven Elastic Resilient
  20. 20. Why Does it Matter? Respond in a deterministic, timely manner. Controls determinism Stays responsive in the face of failure – even cascading failures Stays responsive under workload spikes Basic building block for responsive, resilient, and elastic systems Responsive Resilient Elastic Message Driven
  21. 21. Practices for Successful Microservices Deployment Topologies Reactive Systems Resilience with Circuit Breakers Asynchronous Communication Standardization
  22. 22. Circuit Breaker Keeps systems responsive under failure Avoids cascading failures Especially with multi-generational downstream services Critical part to keeping your 1000 services alive
  23. 23. Practices for Successful Microservices Deployment Topologies Reactive Systems Resilience with Circuit Breakers Asynchronous Communication Standardization
  24. 24. Practices for Successful Microservices Deployment Topologies Reactive Systems Resilience with Circuit Breakers Asynchronous Communication Standardization
  25. 25. Standardization • Monitoring • Need to collect metrics, consistently • Logging • Correlation across services • Uniformity in logs • Security • Need to apply standard security configuration • Environment Resolution • Staging, production, etc. Consistency in the face of Heterogeneity
  26. 26. Standardized Reactive Platform
  27. 27. Akka, Spray, Akka Http & Streams Asynchronous High Performance Resilience & Supervision Great Libraries for building Reactive Systems
  28. 28. Bootstrap and Lifecycle Management Unicomplex: Lightweight bootstrap module Emits lifecycle events: starting, active, stopping Startup and shutdown hooks Allows obtaining the current state
  29. 29. Listener • Declares configuration for port binding, interfaces, security, etc
  30. 30. Service • Akka Http/Spray Routes and Http Request Handler Actors • Configured in squbs-meta.conf • A service can be defined in a dependency artifact
  31. 31. Extension • To start low level (non-actor) facilities needed for the environment
  32. 32. Request/Response Pipeline
  33. 33. Cubes Another deployment Topology squbs: rhymes with cubes Drop-in modules Cubes can run in isolation as well as on a flat classpath Easy to compose/decompose/refactor Cubes share the actor system Provide better predictability
  34. 34. Orchestration task1 task2 task3 task4 task5 Input Output
  35. 35. val task1F = doTask1(input) val task2F = doTask2(input) val task3F = (task1F, task2F) >> doTask3 val task4F = task2F >> doTask4 val task5F = (task3F, task4F) >> doTask5 for { result <- task5F } { requester ! result context.stop(self) } Orchestration task1 task2 task3 task4 task5 Input Output
  36. 36. Orchestration DSL High-performance asynchronous orchestration Responsive: Respond within SLA, with or without results Streamlined error handling Reduced code complexity
  37. 37. More Utilities • Http Client • Admin Console • Actor Registry • Perpetual Stream • Persistence Buffer • …
  38. 38. Summary • Large number of services have benefits, but are more difficult • Control your service topology for more determinism and lower latency • Rule of thumb: No more than two hops of synchronous calls from edge • Reactive systems – ideal for services • Responsive & resilient • Standardization • Walk like a duck, quack like a duck, and manage it like a duck • squbs: Have the cake, and eat it too
  39. 39. Q&A – Feedback Appreciated

×