Downloaded 11 times
![Charts: deployment.yaml
kind: Deployment
spec:
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
command: ["{{ .Values.image.command }}"]
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- containerPort: {{ .Values.service.internalPort }}
volumeMounts:
- name: config-volume
mountPath: /var/config.json
subPath: config.json
livenessProbe:
httpGet:
path: /api/status/
port: {{ .Values.service.internalPort }}
readinessProbe:
httpGet:
path: /api/status/
port: {{ .Values.service.internalPort }}
resources:
{{ toYaml .Values.resources | indent 12 }}
volumes:
- name: config-volume
configMap:
name: {{ .Release.Name }}
26](https://image.slidesharecdn.com/talkfinalgids-orchestrationwithk8s-190421120229/85/Orchestration-with-Kubernetes-26-320.jpg)
Uploaded byKunal Kerkar
Orchestration with Kubernetes
Kubernetes can be used to orchestrate an entire software development pipeline from code to production. A pipeline can include stages for building container images, testing, deploying to staging, load testing, and monitoring. Charts are used to package Kubernetes resources for deployment. Helm is used to deploy charts to clusters for staging and production. Load testing tools like K6 help determine system capacity before production. The pipeline provides fast deployments, failure debugging, and improves developer productivity. Future improvements include integrating Git, using Operators, and improving secret management.
More Related Content
Similar to Orchestration with Kubernetes
Orchestration with Kubernetes
- 1. ORCHESTRATION WITH KUBERNETES Kunal Kerkar SoftwareEngineer, Karix @tsudot 1
- 2. ps://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-pe CLOUD NATIVE CNCF ‣ 200%increase since Dec ‘17 Kubernetes ‣40% (5000+) enterprises 2
- 3. ‣Kubernetes Objects ‣Build Pipeline ‣Packaging ‣Deployment ‣LoadTesting ‣Dev Productivity ‣Areas of Improvement AGENDA 3
- 4.
- 5.
- 6.
- 7.
- 8.
- 9. Developer Github Jenkins Container Registry Chart Repository Production Deploy Staging Deploy LoadTesting Monitoring Pipeline 9
- 10. Pipeline Features 1. Fastdeployments 2. Debugging failures 3. Notifications 4. Immutable builds 10
- 11. Source code 1. Dockerfile 2.Makefile 3. Jenkinsfile 4. charts/ 11
- 12. Developer Github Jenkins Container Registry Chart Repository Production Deploy Staging Deploy LoadTesting Monitoring Pipeline Progress… 12
- 13.
- 14. Tungsten: Dockerfile FROM gcr.io/karix/golang:1.9 ENVGOBIN /go/bin COPY . /go/src/github.com/karixtech/tungsten/ WORKDIR /go/src/github.com/karixtech/tungsten/ RUN dep ensure EXPOSE 8009 EXPOSE 8010 RUN make build 14
- 15. Tungsten: Makefile .PHONY: buildinstall test build: go build -o tungsten-server install: go install test: go test -v ./... 15
- 16. Tungsten: Jenkinsfile 1. Writtenin groovy 2. kubernetes-plugin 3. SemVer build trigger 16
- 17. Developer Github Jenkins Container Registry Chart Repository Production Deploy Staging Deploy LoadTesting Monitoring Pipeline Progress… 17
- 18. Tungsten: Jenkinsfile node(label) { container(‘jenkins-slave'){ stage("Fetch dependent images") { sh("gcloud docker -- pull gcr.io/karix/golang:1.9-2") } } } 18
- 19. Jenkinsfile: Build Image stage("Buildimage") { sh("docker build -f Dockerfile -t ${imageTag} .”); sh("docker build -f postgres/Dockerfile ."); } 19
- 20. Jenkinsfile: Test Image stage("Testimage") { result = sh ( script: "docker run --name ${appName} --network ${appName}-n /bin/bash -c "make test"", returnStatus: true ) } 20
- 21. Developer Github Jenkins Container Registry Chart Repository Production Deploy Staging Deploy LoadTesting Monitoring Pipeline Progress… 21
- 22.
- 23. Charts tungsten ➜ charts git:(master)✗ tree . └── tungsten ├── Chart.yaml ├── config │ ├── prod_config.json │ └── stage_config.json ├── templates │ ├── configmap.yaml │ ├── deployment.yaml │ ├── hpa.yaml │ └── service.yaml └── values ├── prod_values.yaml └── stage_values.yaml 4 directories, 9 files 23
- 24. Helm 1. Tiller (Server) 2.Helm client > https://github.com/helm/ 24
- 25. Charts: configmap.yaml apiVersion: v1 kind:ConfigMap metadata: name: {{ .Release.Name }} data: config.json: |- {{.Files.Get .Values.configFile | printf "%s" | indent 4}} 25
- 26. Charts: deployment.yaml kind: Deployment spec: containers: -name: {{ .Chart.Name }} image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" command: ["{{ .Values.image.command }}"] imagePullPolicy: {{ .Values.image.pullPolicy }} ports: - containerPort: {{ .Values.service.internalPort }} volumeMounts: - name: config-volume mountPath: /var/config.json subPath: config.json livenessProbe: httpGet: path: /api/status/ port: {{ .Values.service.internalPort }} readinessProbe: httpGet: path: /api/status/ port: {{ .Values.service.internalPort }} resources: {{ toYaml .Values.resources | indent 12 }} volumes: - name: config-volume configMap: name: {{ .Release.Name }} 26
- 27. Charts: service.yaml apiVersion: v1 kind:Service spec: type: {{ .Values.service.type }} ports: - port: {{ .Values.service.externalPort }} targetPort: {{ .Values.service.internalPort }} protocol: TCP name: {{ .Values.service.name }} selector: app: {{ template "name" . }} release: {{ .Release.Name }} 27
- 28. Charts: hpa.yaml apiVersion: autoscaling/v2beta1 kind:HorizontalPodAutoscaler spec: scaleTargetRef: apiVersion: apps/v1beta1 kind: Deployment name: {{ template "fullname" . }} minReplicas: {{ .Values.hpa.minReplicas }} maxReplicas: {{ .Values.hpa.maxReplicas }} metrics: - type: Resource resource: name: cpu targetAverageUtilization: {{ .Values.hpa.cpu.t 28
- 29. Charts: Packaging ‣ Generatedfrom values ‣ Packaged into tgz ‣ Push to chart repository (https://github.com/helm/chartmuseum) 29
- 30. Developer Github Jenkins Container Registry Chart Repository Production Deploy Staging Deploy LoadTesting Monitoring Pipeline Progress… 30
- 31.
- 32. Helm <command> 1. Updatelocal repo cache helm repo update 2. Install package helm install -n tungsten stage/tungsten --version 2.13.0 3. Upgrade package helm upgrade tungsten stage/tungsten --version 2.15.0 32
- 33.
- 34.
- 35.
- 36.
- 37. Developer Github Jenkins Container Registry Chart Repository Production Deploy Staging Deploy LoadTesting Monitoring Pipeline Progress… 37
- 38. Kubewatch Source: https://kubernetes.io/ > https://github.com/bitnami-labs/kubewatch/ 1.Project by bitnami 2. Publishes cluster events 38
- 39.
- 40. K6 > https://github.com/loadimpact/k6 k6 run-u 3 -i 10000 sms.js 40
- 41. Developer Github Jenkins Container Registry Chart Repository Production Deploy Staging Deploy LoadTesting Monitoring Pipeline Progress… 41
- 42.
- 43. Developer Github Jenkins Container Registry Chart Repository Production Deploy Staging Deploy LoadTesting Monitoring Pipeline Progress… 43
- 44.
- 45. Developer upgrade Source: https://kubernetes.io/ 1.Quick Deployments 2. Control on cluster 3. Test Coverage 4. Boost Confidence 45
- 46.
- 47. Areas of Improvement Source:https://kubernetes.io/ 1. Gitkube 2. Operators 3. Secrets/Vault 47
- 48.
Editor's Notes
- #3 CNCF survey - cloud native technologies - over 200% - December 2017 - 40% enterprises use k8s in production. Karix.io leverages k8s; send and receive messages over channels; Build and orchestrate; focus deployment pipeline; ensure staging production clusters replica; boost dev productivity - improve system reliability.
- #4 K8s objects; build pipelines, packaging, deployment; throw light optimised CPU and memory by load testing; improved dev productivity - currently working on.
- #5 State machine; set desired state - current state matches desired state; Nodes - VMs & bare metal; master controls each node; kubelet daemon - containers are running in pod
- #6 Pod - basic building block; running process; never standalone pods - cannot have state; dont get rescheduled.
- #7 Manage pods by Deployment; collection similar pods; maintain certain number pods
- #8 Service - abstraction logical set of pods - policy to access them; Load balancers; expose external network - job of balancing traffic
- #9 Build tech platform; integrate existing services - presence web mobile; Responsibility - enabling - developer tooling; goal - setup testing environment; running on VM; integration testing - staging VM ansible; Problem - one environment - QA/acceptance testing; big hurdle - major bottleneck - developer productivity; Realised - could not - single QA - meet deadlines; Goal clear - couldn’t timeshare; on demand; containers got drilled; Karix.io; decided to run containers; architecture - what microservices - independently scalable components; 2 engineers; thought process - developer time - building service - not managing infrastructure; super simple onboarding; Any product - start build pipeline; core unit - all software releases - revolve.
- #10 This is build pipeline that we worked out for our platform karix.io. I will keep coming back to this slide as we progress through our pipeline stages.
- #11 Time gap - release tag - run in production - optimized; build fails - debug the build; notified early on; container building - packaging - pushing notified; Build immutable - release gets tagged, new changes cannot be pushed - cluster in the state; Chose Jenkins - versatile tool; Jenkins Pipeline - DSL - modeling delivery pipeline; Code shipped instructions; turn concept dev-ops into reality; Service will be built - run test cases - find config - how to package; enables developers - part of their code.
- #12 Golden rule - 3 files - folder charts.
- #13 So behind the scenes our developer has pushed their code to github.
- #14 Tungsten - golang; main layers - messaging architecture
- #15 Dockerfile - generic; host base image - high security
- #16 Makefile - compiling, building, test; standard targets - little change - build script;
- #17 Jenkinsfile - groovy; calls standard Makefile - logic upload builds; runs on k8s - spawned pod using the jenkins slave base image; triggered - spawn slave Pod - run instructions - terminate job finishes; kubernetes-plugin; semantic versioning; patch staging; minor major production; release tag as version.
- #18 At this point a release tag has been created and GitHub has notified Jenkins.
- #19 Stages - Jenkinfile; split different stages - debug better; first stage fetch golang
- #20 Build tungsten image; postgres image;
- #21 test cases pass - push to registry
- #22 So back to our pipeline progress, At this point image has been built and pushed to the the container repository.
- #23 Builds - staging, production; charts folder - helm package; templates spew k8s objects
- #24 templates - cm, dep, hpa, svc; config folder - config files; values - template values - env; specific to helm
- #25 tiller; helm-client
- #26 config files mounted; varies env; CM picks config file; location in values.yml; mounted through CM
- #27 Deployment - config files; path to container image - command; readiness/liveness - exposed TCP/HTTP
- #28 Service - route traffic; pod selector - route traffic to; external port - accept incoming traffic; target port
- #29 HPA - auto scaling spec; top of deployment
- #30 Generated from values.yml; helm package tar; tar pushed chart repository; chartmuseum
- #31 At this point the deployment package has been created and pushed to the the chart repository.
- #32 Believed deploying software - breeze; click button - single command; building packaging - get back shell. Jenkins notify built, packaged, pushed to chart repo. Deployment manual
- #33 Pushing versioned chart - update local index; install command; helm fetches the tungsten chart ; deploys CM, SVC and deployment. Rolling updates/rollbacks benefits deploying containers; devops teams - zero downtime application upgrades; overview upgrade works
- #34 tungsten pods; helm terminate signal - start booting new pod. Service incorporate - start sending traffic - readiness checks. Upgrade one by one
- #35 tungsten pods; helm terminate signal - start booting new pod. Service incorporate - start sending traffic - readiness checks. Upgrade one by one
- #36 tungsten pods; helm terminate signal - start booting new pod. Service incorporate - start sending traffic - readiness checks. Upgrade one by one
- #37 tungsten pods; helm terminate signal - start booting new pod. Service incorporate - start sending traffic - readiness checks. Upgrade one by one
- #38 At this point the package has been installed on our staging cluster and is ready for load testing.
- #39 Deployments tracked; cannot be blind; kubewatch; bitnami - publishes notifications; changes upgrades deployments cluster.
- #40 launched karix.io - New to k8s; k8s production; load test feasibility setup - determine resources - individual pods; business requirement to benchmark capacity; came up with average spend on infrastructure vs QPS.
- #41 k6 - seamless integration with pipeline; prometheus
- #42 At this point the load testing has happened.
- #43 resource usages hit threshold - wrt - requests fired on component; trial and error; .5 CPU started benchmarking from there; came up with data no of requests wrt given resources
- #44 And finally the package gets deployed on production with the right set of resources.
- #45 deployment process established - improved dev productivity - efficient workflow
- #46 Quick uniform deployments; feature - existing bug isolated, independent clusters; feature specific cluster - without affecting staging dev; Own environment - deploy builds; low resources - financially sustainable; replica - without compatibility; Test coverage 80% = no fault code; earlier bug the better; multiple envs - bugs reduce drastically; 2 engineers - hired first engineer; deploys to production - first week; Boost confidence - sense of achievement.
- #47 k8s - rapidly developing - incorporate best practices - deploying orchestrating applications - different aspects; known technical debts - ship/release faster; improving and fixing pipeline - specific topics.
- #48 Code changes - rigmarole; “my code is building” - high debugging cycles. Gitkube - runs git remote - push changes; has instructions; Operator - extend type of applications - need to maintain state; database/cache managed - didn’t want operational overhead - persistent storage; operators evolving - explore mechanism - staging/dev; Exploring integration with vault/k8s secrets - challenging part - app needs code changes
- #49 Pulling code - restarting daemons; Ansible scripts; clusterssh - parallel ssh - deployment script; k8s - not one-stop solution; rethink your philosophy; onus of uptime scaling k8s master - desired state always maintained.