Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DevOps Days Boston 2017: Real-world Kubernetes for DevOps

886 views

Published on

DevOps Days Boston 2017

Microservices is an increasingly popular approach to building cloud-native applications. Dozens of new technologies that streamline adopting microservices development such as Docker, Kubernetes, and Envoy have been released over the past few years. But how do you actually use these technologies together to develop, deploy, and run microservices?

In this presentation, we’ll cover the nuances of deploying containerized applications on Kubernetes, including creating a Kubernetes manifest, debugging and logging, and how to build an automated continuous deployment pipeline. Then, we’ll do a brief tour of some of the advanced concepts related to microservices, including service mesh, canary deployments, resilience, and security.

Published in: Software
  • Be the first to comment

DevOps Days Boston 2017: Real-world Kubernetes for DevOps

  1. 1. Kubernetes in about 45 minutes Everything you need to know to be dangerous Philip Lombardi Platform Engineer
  2. 2. datawire.io Who is this Phil Lombardi guy? Twitter: @TheBigLombowski 2
  3. 3. datawire.io Why are we all here? ● You are curious about Kubernetes (and friends) and want a primer! ● You are invested in Kubernetes but are looking to learn about some techniques and tools to make your developers lives better. ● This is the last preso and you feel guilty about leaving early :) 3
  4. 4. datawire.io Agenda... ● Part 1: Containers and Docker and Kubernetes, Oh my! ● Part 2: Kubernetes Core Concepts ● Part 3: Development Workflow ● Part 4: Logging, Debugging and Resiliency ● Q & A 4
  5. 5. datawire.io Lesson 1: Containers and Docker and Kubernetes, Oh my! 5
  6. 6. datawire.io What is a container? ● Lightweight Linux environment. It is a form of virtualization… but very different from a full virtual machine. ● Immutable, deployable artifact. ● Runnable. ● Popularized by Docker but there are many implementations (e.g. LXC, Rkt). 6
  7. 7. datawire.io What is Docker? ● A tool, ecosystem and platform for building, pushing and running containers. ● The most popular container runtime currently. ● Default container runtime in Kubernetes. 7
  8. 8. datawire.io Why Containers? ● Easy and fast to produce. ● Great way to isolate different components in a complex system. ● Ensures a reproducible runtime for your app along the dev -> build -> test -> prod pipeline. ● Easy to share in a team or with external partners. 8
  9. 9. datawire.io What is Kubernetes? ● Runs massive numbers of containers based on lessons learned by Google. ● Schedules and runs all kinds of containers ○ long-lived (e.g. services) ○ short-lived (e.g. pre-launch hooks, cronjobs etc) ● Kubernetes can be thought of as a Distributed OS or process manager 9
  10. 10. datawire.io The Office Tower Analogy Your product is the building as a whole. Your business logic is the offices and workers 10
  11. 11. datawire.io The Office Tower Analogy Kubernetes provides the infrastructure to build your app around. 11 It is the foundational app platform for your team to build your businesses apps around.
  12. 12. datawire.io Why Kubernetes? It is not the only kid in the neighborhood… ● Amazon ECS ● Docker Swarm ● Hashicorp Nomad ● Apache Mesos 12
  13. 13. datawire.io Why Kubernetes? Three big reasons to use Kubernetes over the other solutions: 1. Biggest ecosystem of the bunch and there is a hugely massive community 2. Runnable just about anywhere: cloud, bare-metal, and engineers laptops. 3. Unprecedented cloud portability. 13
  14. 14. datawire.io Kubernetes Architecture Types of nodes: Masters and Workers 14 Docker Kubelet Kubeproxy Kubernetes Node Docker Kubelet Kubeproxy Kubernetes Node Docker Kubelet Kubeproxy Kubernetes Node Etcd API Server Controller Manager Kubernetes Master Scheduler
  15. 15. datawire.io Lesson 2: Core Kubernetes Concepts 15
  16. 16. datawire.io The “Big Five” of Kubernetes Concepts ● Pods ● Deployments ● Services ● ConfigMaps ● Secrets 16
  17. 17. datawire.io A Pod you say? ● One or more strongly-related containers… 17 Story Server name: blog Redis Comment Server Frontend host: kube-worker-0
  18. 18. datawire.io A Pod you say? ● Containers in a pod share the same host, pod IP and port space. 18 IP: 100.124.71.175 Story Server name: blog Redis Comment Server Frontend host: kube-worker-0
  19. 19. datawire.io A Pod you say? ● Unit of scaling is a Pod 19 IP: 100.124.71.175 blog-0 kube-worker-0 IP: 100.124.71.176 blog-1 kube-worker-1 Kubernetes cluster
  20. 20. datawire.io Pods Summary ● Like a host. All containers inside of a pod are run on the same underlying worker machine. ○ Can therefore reference localhost ○ … or share the filesystem ○ … or use unix domain sockets ● All containers in a Pod share the same IP and port space. ● Pods are not durable. ● Pods are a very low-level primitive construct. Necessary to know, but not commonly used directly. 20
  21. 21. datawire.io Deployments ● Simple mechanism to configure, scale and update applications. ● Kubernetes does the rest of the hard work of scheduling the Pods across the cluster to meet desired capacity numbers. ● Works like a thermostat… Ensures the current state is always consistent with the desired state. 21
  22. 22. datawire.io Services ● Services are stable “names” in Kubernetes that enable you to route traffic to Pods across the entire cluster. ● Every service gets its own IP address. ● Services route traffic to pods by matching labels. 22
  23. 23. datawire.io Services Illustrated How to talk to both apps despite different IP addresses? 23 IP: 100.124.71.175 blog-0 kube-worker-0 IP: 100.124.71.176 blog-1 kube-worker-1 Kubernetes cluster
  24. 24. datawire.io Services Illustrated Add a Service which becomes a DNS A record pointing the pod IP addresses 24 IP: 100.124.71.175 blog-0 kube-worker-0 IP: 100.124.71.176 blog-1 kube-worker-1 Kubernetes cluster blog DNS (short) => blog DNS (long) => blog.default.cluster.local
  25. 25. datawire.io Services Illustrated You can have multiple services target pods using labels and selectors. 25 app=blog env=prod blog-0 kube-worker-0 app=blog env=prod blog-1 kube-worker-1 Kubernetes cluster blog blog-staging app=blog blog-0 kube-worker-1 app=blog env=prod app=blog
  26. 26. datawire.io Service Flavors ● Many different flavors of “Service” in Kubernetes ○ ClusterIP ○ NodePort ○ LoadBalancer ○ ExternalName - often forgotten, but very useful! 26
  27. 27. datawire.io Services Summary ● Creates DNS A records pointing at Pod IP addresses ● Powerful label matching capabilities that enable you to route traffic to particular pods (e.g. for blue-green or canary releases). ● Supports DNS SRV records so you can avoid hard coding port numbers in your app code as well. 27
  28. 28. datawire.io ConfigMap ● Containers are immutable… so how do you provide runtime configuration to them? ● Age old question for immutable infrastructure lots of good (and bad) solutions have been built over the years. ● Kubernetes solution is built-in as the ConfigMap. Inject configuration information as… ○ Environment variables ○ Volumes 28
  29. 29. datawire.io Secret ● Cousin of the ConfigMap ● Operates almost exactly the same as a ConfigMap but designed for storing sensitive information. ● Secret information only sent from master to worker nodes when needed by a pod. The data lives in memory so it is not on the disk. ● One important aspect of secrets… the master currently stores them in plaintext. Work in Progress to eliminate this in the future, but worth being aware of. 29
  30. 30. datawire.io Kubernetes to AWS 30 Kubernetes AWS Pod EC2 Instance Deployment AutoScaling Group + Launch Configuration Service ELB ConfigMap N/A Secret N/A
  31. 31. datawire.io Lesson 3: Developer Workflow 31
  32. 32. datawire.io Developers... ● Part of our role involves aiding Developers and making them faster and more productive. ● Kubernetes is awesome and it comes with a lot of power. ● Great power comes with lots of potential for learning pain. ● How do we make developers productive? 32
  33. 33. datawire.io Manifests ● A declarative YAML/JSON config format that describe at a high level how Kubernetes should operate. ● Kubernetes operates like a thermostat. Transforms current state -> desired state based on config in the Manifest. ● Manifests often need to be parameterized (e.g. to change the container image). General approach is to use some kind of templating. 33
  34. 34. datawire.io Structuring an Application ● Often asked, all this stuff is cool, but how should we structure our apps to be consistent and compatible with tooling? ● Strongly recommend a k8s/ directory in the top of your project ○ Manifests can be concrete and ready to use by just running `kubectl apply -f k8s/ ` ○ … or you can take an alternate approach and put templates in that directory and do some kind of config generation with say Python + Jinja2 (for example). 34
  35. 35. datawire.io Writing Manifests ● Avoid hard coding environment configuration into manifests. ○ Templating ○ Kubectl switches (e.g. for namespaces) ● Stick to YAML even if Kubernetes supports JSON… not uncommon to want comments in the manifests. ● Kubernetes manifests can be spread across multiple files or kept in a single file. Strongly recommend using a single file until it becomes bothersome to maintain. 35
  36. 36. datawire.io Development Workflows ● No single workflow that works for all developers or teams ● Need tools that can adapt to changing requirements and process ● Personally, great success with: ○ Trunk-based development model. ○ Using parameterized templates in k8s/ directory. ○ monorepo or “pseudo-monorepo”. ○ Dev-tooling the focuses on speed and maintaining fast iteration cycles 36
  37. 37. datawire.io Forge (https://forge.sh) ● Build and deploy Kubernetes-based microservices quickly. ● Can deploy 1 or 100 services from source to Kubernetes in seconds. ● Changes are applied incrementally. Computes the diff of a change and then pushes the update to Kubernetes. 37
  38. 38. datawire.io Lesson 4: Logging, Debugging and Resiliency 38
  39. 39. datawire.io Logging... ● Kubernetes has a built-in log aggregation but it is limited (STDOUT, STDERR only). ● `kubectl logs` is good enough for devs but invest in a real logging solution for prod. ● You will want something like fluentd and elasticsearch because Kubernetes does not track historical logs for crashed or terminated pods. ○ Also search capabilities are limited to how much of a grep wizard you are. 39
  40. 40. datawire.io Logging... ● There is more to logging than just application logs. ● Consider introducing a service mesh to your cluster that allows you to do per-request logging and tracing. 40
  41. 41. datawire.io Services Mesh ● This was covered in a presentation yesterday so here is the recap: A dedicated infrastructure layer for making service-to-service communication safe and reliable. ● Kubernetes and CNCF are really pushing Lyft Envoy as the mechanism to build a service mesh. 41
  42. 42. datawire.io Simple Log Query Tools ● Because the kubectl logs command is so limited many tools have been written to make it easier... ○ ktail https://github.com/atombender/ktail ○ kubetail https://github.com/johanhaleby/kubetail ○ stern https://github.com/wercker/stern 42
  43. 43. datawire.io Debugging... Kubernetes IS complex. There are a lot of failure scenarios in all kinds of places. The Kubernetes docs are pretty helpful for doing some troubleshooting Application Troubleshooting: https://kubernetes.io/docs/tasks/debug-application-cluster/debug-application/ Cluster Troubleshooting: https://kubernetes.io/docs/tasks/debug-application-cluster/debug-cluster/ 43
  44. 44. datawire.io Debugging The Cool Way ● Sometimes you need something more powerful than logs… ● Or your developers use a shared development-staging cluster and something is broken... ● Classic problem with building web services… How do I attach a local process to the running cloud environment? What about a debugger? ● Super easy in Kubernetes with a tool called Telepresence! http://www.telepresence.io/ 44
  45. 45. datawire.io ● Telepresence proxies network requests, environment variables, and volumes to local Telepresence client ● Code locally on your laptop using your favorite editor and your local filesystem :) Telepresence 45
  46. 46. datawire.io Wrapping Up ● Kubernetes is awesome! ● There’s a lot of power and flexibility ● We need to empower developers by providing them excellent tools that make their lives easier! ● As infrastructure and ops engineers we need to build a stable platform that developers can use without feeling restricted. The service mesh makes this easier. 46
  47. 47. datawire.io Preso Over! Thank you! ● If you’re building cloud applications on top of Kubernetes, check out our open source tools: ● Contact us @datawireio or hello@datawire.io 47

×