ОЛЕКСАНДР ЛИПКО «Graceful Shutdown Node.js + k8s» Online WDDay 2021 https://wdday.org/ Facebook: https://www.facebook.com/wdday.org Linkedin: https://www.linkedin.com/company/wdday

1. Graceful shutdown
how it works inside k8s

2. ● node.js app in k8s cluster
● load ~10-200 rps
● autoscaling
● deployment every 4 hours
Input

3. Problems

4. Errors spike

5. Not consistent state
async createUser(args) {
const { someImportantValue } = await externalService.createUser(args);
const user = await db.Users.create({ ...args, someImportantValue });
return result;
}

6. Zombie connections

7. Reasons

8. 1. We don’t know how k8s works with our app
Reasons

9. How k8s creates pods?
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
containers:
- name: web
image: node
ports:
- name: web
containerPort: 80

10. How k8s creates pods?
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
containers:
- name: web
image: node
ports:
- name: web
containerPort: 80

11. How k8s creates pods?

12. How k8s creates pods?

13. How k8s creates pods?

14. Service
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
ports:
- port: 80
targetPort: 3000
selector:
name: app

15. Service
- k8s finds all the Pods that have
the same label as the selector
and collect their ips
- Store Endpoint record [ip:port]
- Refresh Endpoint every time when
pods in the service
createdelete or modify label

16. Probes
- StartupProbe
Indicates whether the application within the container is started.
- LivenessProbe
Indicates whether the container is running
- ReadinessProbe
Indicates whether the container is ready to respond to requests.

17. Probes
livenessProbe:
path: '/live'
port: 3000
periodSeconds: 10
failureThreshold: 1
startupProbe:
path: '/live
port: 3000
periodSeconds: 5
failureThreshold: 10
readinessProbe:
path: '/ready'
port: 3000
periodSeconds: 10
failureThreshold: 1

18. - Kube-proxy create
iptable rules on each
nodes
- The Ingress routes
external traffic into the
cluster
Where is the external traffic??
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
spec:
rules:
- http:
paths:
- backend:
service:
name: my-service
port:
number: 80
path: /
pathType: Prefix

19. - The ingress subscribe to
the service and check
Endpoint records

20. - Apply pod manifest
- Store the Pod in etcd (Control plane)
- The Kubelet is notified of a new and scheduled Pod
- CRI, CRN, CSI
- The kubelet reports the IP address to the control plane
- Apply service manifest
- The kubelet waits for a successful Readiness probe
- The Endpoints add a new endpoint (IP address + port pair) to
their list
- The Ingress routes traffic to the new IP addresses

21. How k8s deletes pods?

22. How k8s deletes pods?

27. What we need to do?
● Listen to SIGTERM signal
● Wait 15 sec
● Wait to ending all of
processing requests
● Close all of connections
process.on('SIGTERM', () => {
sleep(15);
await waitActiveHandlers()
await db.close();
process.exit(0)
});

28. What we need to do? (2)
● Handle Unhandled Errors and
Crash Events
● Set Readiness probe to 400
● Wait to k8s checking probe
● Gracefully shutdown
process.on('uncaughtException', error => {
await setProbeNotReady();
sleep(15);
await waitActiveHandlers()
await db.close();
process.exit(1);
});

29. Questions?

30. Links
● Zero downtime deployments in Kubernetes
● Configure Liveness, Readiness and Startup Probes
● Lightship (node.js lib)
● K8s best practices from Google