Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Why should i care about stateful containers?

700 views

Published on

Presented by Luke Marsden at Software Circus Amsterdam
Microservices are smashing monolithic databases into lots of pieces. CI and CD is making testing those consistently more and more challenging. This talk will explore the problem space and dive into detailed examples, exploring the pros and cons of both ephemeral data stores and storage orchestration.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Why should i care about stateful containers?

  1. 1. Why should I care about stateful containers? Luke Marsden CTO, ClusterHQ @lmarsden @clusterhq
  2. 2. Microservices are smashing up monolithic databases Many more database instances, many more flavours prod staging in cloud 2 DR in cloud 3 hosted CI dev laptop
  3. 3. Aren’t containers stateless? 12 factor apps
  4. 4. One platform
  5. 5. Dev/prod parity
  6. 6. Cloud services are effectively closed source
  7. 7. Portability
  8. 8. OK, who’s in?
  9. 9. Cattle vs. pets Deep dive
  10. 10. compute local storage infrastructure evolution 1990s servers with disks in Servers Volumes Pets Pets Cattle Pets Cattle Cattle
  11. 11. SAN storage VM VM VM infrastructure evolution 2000s vol vol vol VMware hosts virtualisation and shared storage Servers Volumes Pets Pets Cattle Pets Cattle Cattle
  12. 12. EBS VM VM VM infrastructure evolution 2010s cloud VMs no longer HA vol vol vol EC2 Servers Volumes Pets Pets Cattle Pets Cattle Cattle
  13. 13. infrastructure evolution 2010s compute local storage on-prem “hyperconverged” or “SDS” eg OpenStack + Ceph or ScaleIO VM vol vol Servers Volumes Pets Pets Cattle Pets Cattle Cattle
  14. 14. infrastructure evolution 2010s compute local storage distributed databases bare metal / cloud VMs + local storage db Servers Volumes Pets Pets Cattle Pets Cattle Cattle db
  15. 15. infrastructure evolution 2010s compute local storage distributed databases on SDS vol vol Servers Volumes Pets Pets Cattle Pets Cattle Cattle db db
  16. 16. infrastructure evolution 2015 compute local storage distributed databases on SDS in container clusters vol vol Servers Volumes Pets Pets Cattle Pets Cattle Cattle db db container scheduler
  17. 17. Servers Volumes What is it? Pets Pets just crap Cattle Pets storage orchestration Cattle Cattle ephemeral stateful containers Pet ~= HA provided by infrastructure/platform Cattle ~= HA provided by application
  18. 18. Pros & cons Storage orchestration • Run any database/stateful app • Storage layer responsible for data resilience • Ops independent of database types • Works within a storage zone Ephemeral stateful containers • Requires distributed database(s) • Database-defined data resilience • Ops requires understanding each data service • Can span storage zones
  19. 19. Ephemeral stateful
  20. 20. Storage orchestration
  21. 21. Why should data be pets when servers are cattle?
  22. 22. Relocatable EBS/SDS/SAN…
  23. 23. Stateless things scale out Stateful things scale up Container can be a shard
  24. 24. You can mix and match stateful + stateless in a container cluster
  25. 25. Use case: rebalance cluster storagev1 compute container host (VM or bare metal) c1 c2 container composer + scheduler + SD v2
  26. 26. Use case: rebalance cluster storagev1 compute container host (VM or bare metal) c1 c2 v2 container composer + scheduler + SD
  27. 27. Use case: drain a node storagev1 compute container host (VM or bare metal) c1 c2 v2 v3 c3 c4 v4 container composer + scheduler + SD
  28. 28. Use case: drain a node storage compute container host (VM or bare metal) v3 c3 c4 v4 v1 c1 c2 v2 container composer + scheduler + SD
  29. 29. Use case: drain a node storage compute container host (VM or bare metal) v3 c3 c4 v4 v1 c1 c2 v2 container composer + scheduler + SD
  30. 30. Use case: HA when node fails storagev1 compute container host (VM or bare metal) c1 container composer + scheduler + SD
  31. 31. Use case: HA when node fails storagev1 compute container host (VM or bare metal) container composer + scheduler + SD
  32. 32. Use case: HA when node fails storagev1 compute container host (VM or bare metal) c1 container composer + scheduler + SD
  33. 33. Use case: speed up distributed database recovery storagev1 compute container host (VM or bare metal) c1 container composer + scheduler + SD c2 container overlay network v2 10.0.0.1 10.0.0.2 c3 v3 10.0.0.3
  34. 34. Use case: speed up distributed database recovery storagev1 compute container host (VM or bare metal) c1 container composer + scheduler + SD c2 container overlay network v2 10.0.0.1 10.0.0.2 c3 v3 10.0.0.3
  35. 35. Use case: speed up distributed database recovery storagev1 compute container host (VM or bare metal) container composer + scheduler + SD c2 container overlay network v2 10.0.0.2 c3 v3 10.0.0.3
  36. 36. Use case: speed up distributed database recovery storagev1 compute container host (VM or bare metal) container composer + scheduler + SD c2 container overlay network v2 10.0.0.2 c3 v3 10.0.0.3 c1 10.0.0.1
  37. 37. Connecting storage to containers
  38. 38. Try Flocker today For stateful microservices luke@clusterhq.com clusterhq.com github.com/clusterhq/flocker We are hiring in Bristol (1 hour hop from Schiphol) + SF Bay Area! clusterhq.com/careers

×