Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Scalable Service Architectures Lessons learned Zoltán Németh Engineering Manager, Core Systems
Agenda  Our scalability experience  What is Scalability?  Requirements in detail  Tips and tools  Extras, Closing rem...
Our experience
Streaming stack
Defining scalability Scalability is the ability to handle increased workload by repeatedly applying a costeffective strate...
Self-contained service  Explicitly declare and isolate dependencies  Isolation from the outside system  Static linking ...
Disposability  Maximize robustness with fast startup and graceful shutdown  Disposable processes  Graceful shutdown on ...
Startup and Shutdown  Automate all the things  Chef  Docker  Gold image based deployment  Immutable  Handling tasks ...
Backing Services  Treat backing services as attached resources  No distinction between local and third party services  ...
Processes, concurrency  Stateless processes (not even sticky sessions)  Process types by work type  We <3 linux process...
Statelessness  Store everything in a datastore  Aggregate data  Chandra  Scalable datastores  Redis  Cassandra  Aer...
Monitoring  Application state and metrics  Dashboards  Alerting  Health  Remove failing nodes  Capacity  Act on tre...
Monitoring  Metrics collecting  Graphite, New Relic  Self-aware checks  Cluster state  Zookeeper, Consul  Scaling de...
Load Balance and Resource Allocation  Load Balance: distribute tasks  Utilize machines efficiently  VM compatible apps ...
Load Balance  DNS or API  App level balance  Uniform entry point or proxy  Balance decisions  Load  Zookeeper state ...
Service Separation  Failure is inevitable  Protect from failing components  Cascading failure  Fail fast  Decoupling ...
Service Separation  Rate limiting  Circuit Breaker pattern  Stop cascading failure, allow recovery  Hystrix  Fail fas...
Extras  Debugging features  Logs  Clojure / JS consoles  Runtime configuration via env  Scaling API  Integrating sev...
Reading  Scalable Internet Architectures by Theo Schlossnagle  The 12-factor App: http://12factor.net/  Carnegie Mellon...
Questions syntaxerror@ustream.tv
Scalable Service Architectures
Scalable Service Architectures
Upcoming SlideShare
Loading in …5
×

Scalable Service Architectures

617 views

Published on

In the past years at Ustream we built several systems which had to be scalable and resilient by themselves. This included automatic monitoring and scaling, protection against overloads, and the like. We learned a lot of valuable lessons on the way, and I would like to share those experiences, with practical examples and open source tools.

Published in: Internet
no profile picture user

  • Be the first to comment

  • Be the first to like this

Scalable Service Architectures

  1. 1. Scalable Service Architectures Lessons learned Zoltán Németh Engineering Manager, Core Systems
  2. 2. Agenda  Our scalability experience  What is Scalability?  Requirements in detail  Tips and tools  Extras, Closing remarks
  3. 3. Our experience
  4. 4. Streaming stack
  5. 5. Defining scalability Scalability is the ability to handle increased workload by repeatedly applying a costeffective strategy for extending a system’s capacity. (CMU paper, 2006) How well a solution to some problem will work when the size of the problem increases. When the size decreases, the solution must fit. (dictionary.com and Theo Schlossnagle, 2006)
  6. 6. Self-contained service  Explicitly declare and isolate dependencies  Isolation from the outside system  Static linking  Do not rely on system packages
  7. 7. Disposability  Maximize robustness with fast startup and graceful shutdown  Disposable processes  Graceful shutdown on SIGTERM  Handling sudden death: robust queue backend
  8. 8. Startup and Shutdown  Automate all the things  Chef  Docker  Gold image based deployment  Immutable  Handling tasks before shutdown
  9. 9. Backing Services  Treat backing services as attached resources  No distinction between local and third party services  Easily swap out resources  Export services via port binding  Become the backing service for another app
  10. 10. Processes, concurrency  Stateless processes (not even sticky sessions)  Process types by work type  We <3 linux process  Shared-nothing  adding concurrency is safe  Process distribution spanning machines
  11. 11. Statelessness  Store everything in a datastore  Aggregate data  Chandra  Scalable datastores  Redis  Cassandra  Aerospike  Handling user sessions
  12. 12. Monitoring  Application state and metrics  Dashboards  Alerting  Health  Remove failing nodes  Capacity  Act on trends
  13. 13. Monitoring  Metrics collecting  Graphite, New Relic  Self-aware checks  Cluster state  Zookeeper, Consul  Scaling decision types  Capacity amount  Graph derivative  App requests
  14. 14. Load Balance and Resource Allocation  Load Balance: distribute tasks  Utilize machines efficiently  VM compatible apps  Flexibility  Adapting to available resources
  15. 15. Load Balance  DNS or API  App level balance  Uniform entry point or proxy  Balance decisions  Load  Zookeeper state  Resource policies
  16. 16. Service Separation  Failure is inevitable  Protect from failing components  Cascading failure  Fail fast  Decoupling  Asynchronous operations  Message queues
  17. 17. Service Separation  Rate limiting  Circuit Breaker pattern  Stop cascading failure, allow recovery  Hystrix  Fail fast, fail silent  Service decoupling
  18. 18. Extras  Debugging features  Logs  Clojure / JS consoles  Runtime configuration via env  Scaling API  Integrating several cloud providers  Automatic start / stop
  19. 19. Reading  Scalable Internet Architectures by Theo Schlossnagle  The 12-factor App: http://12factor.net/  Carnegie Mellon Paper: http://www.sei.cmu.edu/reports/06tn012.pdf  Circuit Breaker: http://martinfowler.com/bliki/CircuitBreaker.html  Release It! by Michael T. Nygard
  20. 20. Questions syntaxerror@ustream.tv

×