Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scalable and Available Services with Docker and Kubernetes

3,003 views

Published on

Breaking down your monolith into microservices — or starting fresh with a microservice architecture on a new project — is only half the battle. The next step is making your containerized microservice application fault tolerant and highly available, but how do you get there? How do you go from your local development machine to a production-grade cluster? See how container orchestration platforms like Kubernetes and Swarm can be essential assets for your production workloads, and learn how to replace bottlenecks and single points of failure with highly available, scalable microservice deployments.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Scalable and Available Services with Docker and Kubernetes

  1. 1. Laura Frank Director of Engineering, Codeship Scalable and Available Services with $CONTAINER_TOOL
  2. 2. R A I N I N G O N YO U R PA R A D E Highly-available applications existed before containers
  3. 3. We love to think we’re solving new problems in new ways
  4. 4. We shouldn’t confuse new tools with new problems
  5. 5. Container tooling has changed the way we design, build, run, and ship applications.
  6. 6. is a new solution for a longstanding problem. Container tooling
  7. 7. Containers aren’t the point We reason about services
  8. 8. Before the late 1980s
  9. 9. 1990s-ish
  10. 10. 3:00am when you’re on call
  11. 11. How can we guarantee availability in an environment that will definitely fail?
  12. 12. DISTRIBUTED APPLICATIONS ENGINEERING, 1998 “Redundancy and recovery are the two main approaches to solve this problem.”
  13. 13. An Imprecise Guideline ignoring many system constraints redundancyrequired (numberofreplicas) time to recover from failure (generic time units)
  14. 14. Container tools have some pretty sweet ways to deal with both redundancy and recovery.
  15. 15. Recovery Control Theory FTW
  16. 16. Your orchestration platform is continuously trying to reconcile actual state with declared state.
  17. 17. Desired State - ClusterOrch actions to converge state Actual State at time T
  18. 18. An Observability Problem If a system can’t be observed, it can’t be controlled.
  19. 19. An Observability Problem Failure Process State User Input
  20. 20. Desired State - ClusterMe! Actual State at time T
  21. 21. An Observability Problem Offloading the responsibility of observability to an orchestrator improves the level of controllability in your system
  22. 22. Atomic Scheduling Units Scheduler Orchestrator taskN task0 task1 Service Spec desired state Service Object actual state
  23. 23. Kubernetes Master Desired State Scheduler Controllers API Server task0 task1 etcd
  24. 24. Kubernetes Master Desired State etcd converged! Scheduler Controllers API Server
  25. 25. Using an orchestration tool, your system never fails… it just doesn’t converge
  26. 26. Redundancy Replicating and scheduling for high availability
  27. 27. HA application problems scheduling problems task scheduling problems
  28. 28. binpack
  29. 29. binpack
  30. 30. spread
  31. 31. spread (optimized for HA apps)
  32. 32. Most modern orchestration systems use an optimized scheduling algorithm for dispatching services across a set of nodes. G R E AT N E W S
  33. 33. It is not your tool’s responsibility to know about your system and business constraints • topology* (some schedulers are topology aware) • specifics like OS, kernel, instance family • PII and other compliance YO U S T I L L H AV E TO D O W O R K
  34. 34. These tools work on the service level, not the infrastructure level R E M I N D E R
  35. 35. Scheduling Constraints Restrict services to specific nodes, such as specific architectures, security levels, or types, first apply a label to the nodes docker service create --constraint 'node.labels.type==web' my-app in Docker
  36. 36. nodeSelector has been around since 1.0, but there are alternatives which are more expressive nodeAffinity has been around since 1.2 (still in beta). nodeAntiAffinity does the opposite — you can repel things from one another. in Kubernetes Scheduling Constraints
  37. 37. requiredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: some-node-label-key operator: Exists in Kubernetes Scheduling Constraints
  38. 38. requiredDuringSchedulingIgnoredDuringExecution in Kubernetes Scheduling Constraints requiredDuringSchedulingRequiredDuringExecution This allows labels to change while the pod is running and won’t result in eviction
  39. 39. Implements a spread strategy over nodes that belong to a certain category. This is a “soft” preference --placement-pref ‘spread=node.labels.key’ in Docker Placement Preferences
  40. 40. preferredDuringSchedulingIgnoredDuringExecution in Kubernetes Placement Preferences
  41. 41. Topology-aware Scheduling us-east-1 us-east-2 us-east-1 us-west-1
  42. 42. Topology-aware Scheduling us-east-1 us-east-2 us-east-1 us-west-1
  43. 43. Topology-aware Scheduling Kubernetes has a topology-aware scheduler! Read the docs. In Docker, apply labels to your nodes, and use a placement preference like: --placement-pref ‘spread=node.labels.region’
  44. 44. An Imprecise Guideline ignoring most constraints redundancyrequired (numberofreplicas) time to recover from failure (hypothetical time units)
  45. 45. The Future of Orchestration Warning: opinions
  46. 46. A Framework for Evaluation Genesis Custom Built Product Commodity Visible (Lots of Management) Invisible (No Management)
  47. 47. Genesis Custom Built Product Commodity Wardley Maps (simplified) Time InvisibleVisible
  48. 48. Genesis Custom Built Product Commodity InvisibleVisible Electricity 18th Century Electricity 19th Century Electricity now
  49. 49. Genesis Custom Built Product Commodity Electricity Compute InvisibleVisible
  50. 50. Genesis Custom Built Product Commodity Container Runtime 2000s Container Runtime 2014-2015 Container Runtime now InvisibleVisible
  51. 51. Genesis Custom Built Product Commodity Container Orchestrator Container Runtime InvisibleVisible
  52. 52. Genesis Custom Built Product Commodity Container Orchestrator Container Runtime InvisibleVisible ? ? ?
  53. 53. Orchestration is becoming commoditized. Orchestrators will not be able to differentiate easily.
  54. 54. C O M M O D I T I Z AT I O N If you have a hand-rolled solution for running apps with containers, it’s safe to migrate to an orchestration platform.
  55. 55. I N N OVAT I O N Solutions to old problems get commoditized, but it leaves room for genesis elsewhere
  56. 56. Genesis Custom Built Product Commodity Container Orchestrator Container Runtime InvisibleVisible ? ? ? Istio & service mesh tools Whatever Heptio is building Storage!
  57. 57. Closing Thoughts
  58. 58. How can we guarantee availability in an environment that will definitely fail?
  59. 59. DISTRIBUTED APPLICATIONS ENGINEERING, 1998 “Redundancy and recovery are the two main approaches to solve this problem.” Google became a company in 1998!
  60. 60. Laura Frank Director of Engineering, Codeship @rhein_wein Thanks!

×