![25Google Cloud Platform
FROM node:alpine
RUN apk update && apk add imagemagick
RUN groupadd -r nodejs
RUN useradd -m -r -g nodejs nodejs
USER nodejs
ADD https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64
/usr/local/bin/dumb-init
RUN chmod +x /usr/local/bin/dumb-init
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ADD package.json package.json
RUN npm install
ADD index.js index.js
CMD npm start
Example Dockerfile](https://image.slidesharecdn.com/workshopkubernetesbestpractices-170801185928/85/Kubernetes-best-practices-25-320.jpg)
![42Google Cloud Platform
apiVersion: v1
kind: Pod
metadata:
name: awesomeapp-pod
labels:
app: awesomeapp
annotations:
pod.beta.kubernetes.io/init-containers: '[
{
"name": "init-myapp",
"image": "busybox",
"command": ["sh", "-c", "until nslookup myapp; do echo waiting for myapp; sleep 2; done;"]
},
{
"name": "init-mydb",
"image": "busybox",
"command": ["sh", "-c", "until nslookup mydb; do echo waiting for mydb; sleep 2; done;"]
}
]'
spec:
containers:
- name: awesomeapp-container
image: busybox
command: ['sh', '-c', 'echo The app is running! && sleep 3600']](https://image.slidesharecdn.com/workshopkubernetesbestpractices-170801185928/85/Kubernetes-best-practices-42-320.jpg)
More Related Content
Similar to Kubernetes best practices
Kubernetes best practices
- 1.
- 2. Google Cloud Platform2 Kubernetes is really flexible
- 3. Google Cloud Platform3 But you might yourself in the
- 4.
- 5. Google Cloud Platform5 Don’t trust arbitrary base images!
- 6. 6Google Cloud Platform StaticAnalysis of Containers https://github.com/banyanops/collector https://github.com/coreos/clair
- 7. Google Cloud Platform7 Use small base images
- 8. 8Google Cloud Platform Overhead Node.jsApp Your App → 5MB Your App’s Dependencies → 95MB Total App Size → 100MB Docker Base Images: node:8 → 667MB node:8-wheezy → 521MB node:8-slim → 225MB node:8-alpine → 63.7MB scratch → ~50MB
- 9. 9Google Cloud Platform Overhead Node.jsApp Your App → 5MB Your App’s Dependencies → 95MB Total App Size → 100MB Docker Base Images: node:8 → 667MB node:8-wheezy → 521MB node:8-slim → 225MB node:8-alpine → 63.7MB scratch → ~50MB ← 6.6x App Size!!
- 10. 10Google Cloud Platform Overhead Node.jsApp Your App → 5MB Your App’s Dependencies → 95MB Total App Size → 100MB Docker Base Images: node:8 → 667MB node:8-wheezy → 521MB node:8-slim → 225MB node:8-alpine → 63.7MB scratch → ~50MB ← 6.6x App Size!! ← 13.3x“min” overhead!!
- 11. 11Google Cloud Platform Overhead Node.jsApp Your App → 5MB Your App’s Dependencies → 95MB Total App Size → 100MB Docker Base Images: node:8 → 667MB node:8-wheezy → 521MB node:8-slim → 225MB node:8-alpine → 63.7MB scratch → ~50MB ← 6.6x App Size!! ← 13.3x“min” overhead!! Pros: Builds are faster Need less storage Cold starts (image pull) are faster Potentially less attack surface Cons: Less tooling inside container “Non-standard” environment
- 12. Google Cloud Platform12 Use the “builder pattern”
- 13. 13Google Cloud Platform Code BuildContainer Compiler Dev Deps Unit Tests etc... Build Artifact(s) Runtime Container Runtime Env Debug/Monitor Tooling Binaries Static Files Bundles Transpiled Code
- 14. Google Cloud Platform14 Docker bringing native support for multi-stage builds in Docker CE 17.05
- 15.
- 16. Google Cloud Platform16 Use a non-root user inside the container
- 17. 17Google Cloud Platform FROMnode:alpine RUN apk update && apk add imagemagick RUN groupadd -r nodejs RUN useradd -m -r -g nodejs nodejs USER nodejs ADD package.json package.json RUN npm install ADD index.js index.js CMD npm start Example Dockerfile
- 18. 18Google Cloud Platform Enforceit! apiVersion: v1 kind: Pod metadata: name: hello-world spec: containers: # specification of the pod’s containers # ... securityContext: runAsNonRoot: true
- 19. Google Cloud Platform19 Make the filesystem read-only
- 20. 20Google Cloud Platform apiVersion:v1 kind: Pod metadata: name: hello-world spec: containers: # specification of the pod’s containers # ... securityContext: runAsNonRoot: true readOnlyRootFilesystem: true Enforce it!
- 21. Google Cloud Platform21 One process per container
- 22. Google Cloud Platform22 Don’t restart on failure. Crash cleanly instead.
- 23. Google Cloud Platform23 Log to stdout and stderr
- 24. Google Cloud Platform24 Add “dumb-init” to prevent zombie processes
- 25. 25Google Cloud Platform FROMnode:alpine RUN apk update && apk add imagemagick RUN groupadd -r nodejs RUN useradd -m -r -g nodejs nodejs USER nodejs ADD https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 /usr/local/bin/dumb-init RUN chmod +x /usr/local/bin/dumb-init ENTRYPOINT ["/usr/bin/dumb-init", "--"] ADD package.json package.json RUN npm install ADD index.js index.js CMD npm start Example Dockerfile
- 26. Google Cloud Platform26 Good News: No need to do this in K8s 1.7
- 27.
- 28. Google Cloud Platform28 Use the “record” option for easier rollbacks
- 29. 29Google Cloud Platform $kubectl apply -f deployment.yaml --record … $ kubectl rollout history deployments my-deployment deployments "ghost-recorded" REVISION CHANGE-CAUSE 1 kubectl apply -f deployment.yaml --record 2 kubectl edit deployments my-deployment 3 kubectl set image deployment/my-deplyoment my-container=app:2.0
- 30. Google Cloud Platform30 Use plenty of descriptive labels
- 31. 31Google Cloud Platform apiVersion:extensions/v1beta1 kind: Deployment metadata: name: web spec: replicas: 12 template: metadata: labels: name: web color: blue experimental: 'true' Labels
- 32. 32Google Cloud Platform App:Nifty Phase: Dev Role: FE App: Nifty Phase: Test Role: FE App: Nifty Phase: Dev Role: BE App: Nifty Phase: Test Role: BE Labels
- 33. 33Google Cloud Platform App:Nifty Phase: Dev Role: FE App: Nifty Phase: Test Role: FE App: Nifty Phase: Dev Role: BE App: Nifty Phase: Test Role: BE Labels App = Nifty
- 34. 34Google Cloud Platform App:Nifty Phase: Dev Role: FE App: Nifty Phase: Test Role: FE App: Nifty Phase: Dev Role: BE App: Nifty Phase: Test Role: BE Labels Role = BE
- 35. 35Google Cloud Platform App:Nifty Phase: Dev Role: FE App: Nifty Phase: Test Role: FE App: Nifty Phase: Dev Role: BE App: Nifty Phase: Test Role: BE Labels Phase = Dev
- 36. 36Google Cloud Platform App:Nifty Phase: Dev Role: FE App: Nifty Phase: Test Role: FE App: Nifty Phase: Dev Role: BE App: Nifty Phase: Test Role: BE Labels Phase = Dev Role = FE
- 37. Google Cloud Platform37 Use sidecar containers for proxies, watchers, etc
- 38. 38Google Cloud Platform Examples App Database localhost AppApp Proxy Proxy Proxy secure connection
- 39. 39Google Cloud Platform Auth,Rate Limiting, etc. Examples App Incoming Requests localhost App App Proxy Proxy Proxy
- 40. Google Cloud Platform40 Don’t use sidecars for bootstrapping!
- 41. Google Cloud Platform41 Use init containers instead!
- 42. 42Google Cloud Platform apiVersion:v1 kind: Pod metadata: name: awesomeapp-pod labels: app: awesomeapp annotations: pod.beta.kubernetes.io/init-containers: '[ { "name": "init-myapp", "image": "busybox", "command": ["sh", "-c", "until nslookup myapp; do echo waiting for myapp; sleep 2; done;"] }, { "name": "init-mydb", "image": "busybox", "command": ["sh", "-c", "until nslookup mydb; do echo waiting for mydb; sleep 2; done;"] } ]' spec: containers: - name: awesomeapp-container image: busybox command: ['sh', '-c', 'echo The app is running! && sleep 3600']
- 43. Google Cloud Platform43 Don’t use :latest or no tag
- 44. Google Cloud Platform44 Readiness and Liveness probes are your friend
- 45. 45Google Cloud Platform Readiness→ Is the app ready to start serving traffic? ● Won’t be added to a service endpoint until it passes ● Required for a “production app” in my opinion Liveness → Is the app still running? ● Default is “process is running” ● Possible that the process can be running but not working correctly ● Good to define, might not be 100% necessary These can sometimes be the same endpoint, but not always Health Checks
- 46.
- 47. Google Cloud Platform47 Don’t always use type: LoadBalancer
- 48. Google Cloud Platform48 Ingress is great
- 49. 49Google Cloud Platform Ingress Service1 Service 2 Service 3 websocket.mydomain.com mydomain.com /foo /bar
- 50. Google Cloud Platform50 type: NodePort can be “good enough”
- 51. Google Cloud Platform51 Use Static IPs. They are free* !
- 52. 52Google Cloud Platform apiVersion:v1 kind: Service metadata: name: myservice spec: type: LoadBalancer loadBalancerIP: QQQ.ZZZ.YYY.XXX ports: - port: 80 targetPort: 3000 protocol: TCP selector: name: myapp $ gcloud compute addresses create ingress --global … $ gcloud compute addresses create myservice --region=us-west1 Created … address: QQQ.ZZZ.YYY.XXX … $ apiVersion: extensions/v1beta1 kind: Ingress metadata: name: myingress annotations: kubernetes.io/ingress.global-static-ip-name: "ingress" spec: backend: serviceName: myservice servicePort: 80
- 53. Google Cloud Platform53 Map external services to internal ones
- 54. 54Google Cloud Platform ExternalServices kind: Service apiVersion: v1 metadata: name: mydatabase namespace: prod spec: type: ExternalName externalName : my.database.example.com ports: - port: 12345 kind: Service apiVersion: v1 metadata: name: mydatabase spec: ports: - protocol: TCP port: 80 targetPort: 12345 kind: Endpoints apiVersion: v1 metadata: name: mydatabase subsets: - addresses: - ip: 10.128.0.2 ports: - port: 12345 Hosted Database Database outside cluster but inside network
- 55.
- 56. Google Cloud Platform56 Use Helm Charts
- 57. Google Cloud Platform57 ALL downstream dependencies are unreliable
- 58. Google Cloud Platform58 Make sure your microservices aren’t too micro
- 59. Google Cloud Platform59 Use a “Service Mesh”
- 60.
- 61. Google Cloud Platform61 Use a PaaS?
- 62.
- 63.
- 64. Google Cloud Platform64 Use Google Container Engine
- 65. Google Cloud Platform65 Resources, Anti-Affinity, and Scheduling
- 66. 66Google Cloud Platform NodeAffinity hostname zone region instance-type os arch custom!
- 67. 67Google Cloud Platform NodeTaints / Tolerations special hardware dedicated hosts etc
- 68. 68Google Cloud Platform PodAffinity / Anti-Affinity hostname zone region
- 69. Google Cloud Platform69 Use Namespaces to split up your cluster
- 70. Google Cloud Platform70 Role Based Access Control
- 71. Google Cloud Platform71 Unleash the Chaos Monkey
- 72. 72Google Cloud Platform MoreResources ● http://blog.kubernetes.io/2016/08/security-best-practices-kubernetes-deployment.html ● https://github.com/gravitational/workshop/blob/master/k8sprod.md ● https://nodesource.com/blog/8-protips-to-start-killing-it-when-dockerizing-node-js/ ● https://www.ianlewis.org/en/using-kubernetes-health-checks ● https://www.linux.com/learn/rolling-updates-and-rollbacks-using-kubernetes-deployments ● https://kubernetes.io/docs/api-reference/v1.6/
- 73. Questions? What best practicesdo you have?