Download to read offline
Uploaded byOleksandr Slynko
Upgrade Kubernetes the boring way
This document outlines the process of upgrading Kubernetes clusters while minimizing downtime and ensuring application availability. It emphasizes key steps such as planning upgrades for etcd, master, and worker nodes, checking cluster health, and preparing backups. Additionally, it provides strategies for managing high availability during upgrades, including using replicas and readiness probes.
More Related Content
Similar to Upgrade Kubernetes the boring way
![[k8s] Kubernetes terminology (1).pdf](https://cdn.slidesharecdn.com/ss_thumbnails/k8skubernetesterminology1-230513211357-e2d31a79-thumbnail.jpg?width=640&height=640&fit=bounds)
Upgrade Kubernetes the boring way
- 1. © Copyright 2018Pivotal Software, Inc. All rights Reserved. Version 1.0 Upgrade Kubernetes the boring way Oleksandr Slynko @alex_sly
- 2. Cloud Foundry ContainerRuntime ● Instantiate, deploy, and manage highly available Kubernetes clusters ● Rolling upgrades
- 3. Cloud Foundry ContainerRuntime ● Instantiate, deploy, and manage highly available Kubernetes clusters ● Rolling upgrades ● We have fancy mascot
- 4. Which version doyou use?
- 5. About 15% ofKubernetes users are 2 major versions behind, and 10% use unsupported versions. According SIG Apps survey
- 6.
- 9. Why people don’tupgrade? ● Don’t know how ● Don’t want to affect applications uptime ● If it works, don’t touch it
- 10. How to upgradeKeep the applications running Prepare to the upgrade
- 11. How to upgradecluster?
- 12.
- 13.
- 14. Plan etcdetcd etcd mastermaster master nodenodenode nodenode node
- 15. Plan 1. Upgrade etcd 2.Upgrade master 3. Upgrade worker
- 16. Plan 1. Upgrade etcd 2.Upgrade master 3. Upgrade worker 0. Backup
- 17. Plan 1. Upgrade etcd 2.Upgrade master 3. Upgrade worker 0. Backup -1. Check cluster health
- 18. Healthcheck kubectl get componentstatus kubectlget nodes kubectl get pods --all-namespaces --field-selector 'status.phase!=Running,status.phase!=Succeeded'
- 19. Backup ● Etcdctl orArk ● Know how to restore backup ● Practice in restoring ● Test your backups
- 20. etcd etcdetcd etcd mastermaster master nodenodenode nodenode node etcd https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/
- 21. Working cluster etcdetcd etcd mastermastermaster nodenode node nodenode node
- 22. Master etcdetcd etcd mastermaster master nodenodenode nodenode node master LOAD BALANCER
- 23. Working cluster inmixed mode etcdetcd etcd mastermaster master nodenode node nodenode node LOAD BALANCER
- 24. Working cluster inmixed mode (mostly) etcdetcd etcd mastermaster master nodenode node nodenode node
- 25. Node nodenode node nodenodenode node
- 26. Node nodenode node nodenodenode node
- 27. nodenode node nodenodenode AZ 1 AZ 2 AZ 3 node
- 28. How to upgradecluster without downtime?
- 29. HA Applications ● Replicas ●Anti-affinity ● Readiness & liveness probes ● Graceful shutdown ● Pre & post stop hooks (ie https://github.com/kubernetes/ingress-nginx/issues/322) ● PodDisruptionBudget
- 30. nodenode node nodenodenode AZ 1 AZ 2 AZ 3 node
- 31. kubectl drain ${node_name}--force --ignore-daemonsets Drain node node node AZ 1 node
- 32.
- 33. Wait for disksto be detached node node AZ 1 node
- 34.
- 35. kubectl delete node"${node_name}" --ignore-not-found Delete the node node node AZ 1 node
- 36.
- 37. etcdetcd etcd mastermaster master nodenodenode nodenode node
- 38. Plan 1. Upgrade etcd 2.Upgrade master 3. Upgrade worker 4. Check cluster health 0. Backup -1. Check cluster health
- 39. “If it hurts,do it more often.”
- 40. How to upgradecluster without downtime continuously?
- 41. ● Test thateverything can be deployed on new version Test deployment
- 42. ✓ All basicKubernetes resources must survive upgrades ✓ CRDs that you use must survive as well ✓ Write sample app to measure downtime ✓ Check that new applications can be deployed on the cluster ✓ Check that settings haven’t changed Test upgrade
- 43. Recap How to upgrade 1.Check the cluster health 2. Backup 3. Upgrade etcd 4. Upgrade master 5. Upgrade worker 6. Check the cluster health Keep the applications running ● Make applications HA ● Cordon and drain nodes ● It takes time Prepare to the upgrade ● Test deployability of a new cluster ● Test upgrades
- 44. ● Automate upgrades ●Do maintenance during working hours ● Don’t write your own installer Bonus
- 45. Transforming How TheWorld Builds Software © Copyright 2018 Pivotal Software, Inc. All rights Reserved.