Download as PDF, PPTX
![Repo .travis.yml
language: go
go:
- '1.16'
services:
- docker
install:
- curl -sL https://${GITHUB_TOKEN}@raw.githubusercontent.com
- if [ -f "tools/travis/install.sh" ]; then bash tools/travi
script:
- dep ensure
- make lint
- make test
- if [ -z "${TRAVIS_TAG}" ]; then make snapshot; fi;
deploy:
provider: script # deploy to docker repo](https://image.slidesharecdn.com/index-181219053047/85/Effective-Building-your-Platform-with-Kubernetes-Keep-it-Simple-51-320.jpg)
![1. CLEAN UP
Single script for repo - Makefile [1]
Resurrect the README
[1] With zsh or bash auto-completion plug-in in your terminal.](https://image.slidesharecdn.com/index-181219053047/85/Effective-Building-your-Platform-with-Kubernetes-Keep-it-Simple-71-320.jpg)
![6. KEEP IT RUNNING
Bridge the new with old:
Use External Services in Kubernetes
Expose k8s in the Legacy [1]
[1] hard coded IP:PORT, considered: K8S events to Consul](https://image.slidesharecdn.com/index-181219053047/85/Effective-Building-your-Platform-with-Kubernetes-Keep-it-Simple-76-320.jpg)
![WHAT DID NOT WORK
1. Too many PoC, cut to 2 weeks max
2. Doing it with too small steps
3. Push back to k8s yaml [*]
4. Alert rules too hard to write
[*] With coaching, I thought, it is OK](https://image.slidesharecdn.com/index-181219053047/85/Effective-Building-your-Platform-with-Kubernetes-Keep-it-Simple-83-320.jpg)
Uploaded byWojciech Barczyński
Effective Building your Platform with Kubernetes == Keep it Simple
The document discusses effective platform building using Kubernetes, focusing on streamlining infrastructure and development practices. It emphasizes continuous deployment, the use of makefiles, and transitioning to microservices while addressing challenges faced during the integration process with legacy systems. Key technologies mentioned include Docker, Prometheus, Grafana, and various cloud services for scalability and management.
More Related Content
Similar to Effective Building your Platform with Kubernetes == Keep it Simple
![Effective Kubernetes == Keep it Simple [Ignite Talk DevOpsDays Warsaw]](https://cdn.slidesharecdn.com/ss_thumbnails/slides-181120223017-thumbnail.jpg?width=640&height=640&fit=bounds)
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
Effective Building your Platform with Kubernetes == Keep it Simple
- 1. EFFECTIVE PLATFORM BUILDING WITHKUBERNETES Wojciech Barczyński - | 18 December 2018 SMACC.io Hypatos.ai
- 2. WOJCIECH BARCZYŃSKI Lead Soware Developer & System Engineer Organizer Golang Warsaw Meetup Visiting Lecturer at WSB and ALK 2019 ➡ Trainings & Consultancy
- 3. STORY Kubernetes + Go+ ... SMACC - Fintech / ML - since 2017 Lyke - Mobile Fashion app - since 2016 I do not like Infra
- 10. KUBERNETES Container management Battery for12factor apps ...heading to integration platform ...becoming a framework for your Xubernetes
- 11. KUBERNETES Kubernetes Node Node Node Node App Ingress Controller Repository make docker_push;kubectl create -f app-srv-dpl.yaml
- 12. SCALE UP! SCALEDOWN! Kubernetes Node Node Node Node App Ingress Controller App Appscale 3x kubectl --replicas=3 -f app-srv-dpl.yaml
- 13. DEPLOYMENT AND PODS Node Master Deployment Dockercontainers Node Docker Containers Pod Schedule Schedule
- 14. SERVICE AND PODS Node Deployment Pod Node Pod services Fixedvirtual address Fixed DNS entry Service matches pods based on labels
- 15. INGRESS Pattern Target AppService api.smacc.io/v1/users users-v1 api.smacc.io/v2/users users-v2 smacc.io web
- 16. BASIC CONCEPTS Name Purpose ServiceInterface Entry point (Service Name) Deployment Factory How many pods, which pods Pod Implementation 1+ docker running
- 17. SERVICE DISCOVERY AND LABELING namesin DNS: curl labels: name=value annotations: prometheus.io/scrape: "true" http://users/list
- 18. SERVICE DISCOVERY loosely couplecomponents auto-wiring with logging and monitoring drop-in installation (traefik, prometheus, ...)
- 19. ALL HAIL THEGIT! Yaml integration: monitoring, alarming, ingress-controller ... Devs can forget about infra... almost DevOps Culture Dream! + all tools -> CRD
- 28. ALL IN GIT smacc-platform.git branches:dev, staging, master Generated k8s files ENV variables for each of environment Similar to the Kelsey Hightower approach
- 29. KUBERNETES Pure and generated fromsimple templates with ${ENV_VAR} 2x kubernetes operators
- 30. LYKE Now JollyChic Indonesia E-commerce Indonesia Mobile-only 50k+users 2M downloads Top 10 Fashion Apps Google Play Store http://www.news.getlyke.com/single- post/2016/12/02/Introducing-the- New-Beautiful-LYKE
- 31. CHALLENGES 50+ VMs inAmazon, 1 VM - 1 App 65% Idle machine $$$ Puppet with manual deployment process Fear, Forgotten components Performance issues
- 32. APPROACH 1. Simplify Infrastructure 2.Change the Development Practices (12factor + kubernetes) 3. Make everybody learn Kubernetes 4. Change the Work Organization see: Conway's law
- 33. SIMPLIFY 1. Kubernetes withGoogle Kubernetes Engine 2. Terraform for all new 3. Database-as-a-service
- 34. SIMPLIFY 1. Prometheus, AlertManager,and Grafana 2. Elasticsearch-Fluentd-Kibana 3. Google Identity-Aware-Proxy to protect all dev dashboards One person efford
- 35. Architecture During Migration Staging K8S 6 Production K8S 10 GoogleLB Google LB Live 30+ Staging 10+ Google AWS Cloudfront (Live/Staging) api.xstaging-api.x Google LBGoogle LBELB Google LBGoogle LBELB VPN VPN lyke.x staging-lyke.x Prometheus EFK - 2 ElasticSearch 4 nodes Lambdas
- 36. Bridge the newwith old Service <External> Rabbit Prometheus Exporter RabbitMQ Google AWS Kubernetes EC2 http://rabbitmq:15672 IP 9090 Monitor legacy with new stack
- 37.
- 38. CONTINUOUS DEPLOYMENT Tests Build docker Deployto Google Container Registry and k8s No config/secrets applied Vault planned; moving secrets to runtime
- 39. GIT REPO |- tools ||- kube-service.yaml | - kube-deployment.yaml | |- Dockerfile |- VERSION - Makefile
- 40. Makefile Copy&Paste from theproject to project SERVICE_NAME=v-connector GCP_DOCKER_REGISTRY=eu.gcr.io test: test_short test_integration run_local: docker_build: docker_push kube_create_config: kube_apply: kube_deploy:
- 41. WHAT WORKED 1. Copy&PasteMakefile and k8s yaml 2. Separate deployments a good transition strategy
- 42. SMACC Machine Learning FinTech SaaSand API platform From Enterprise (Deutsche Bank, AoK) to SME Well-known FinTech Startup in Germany
- 44. GET TO K8S Legacyon AWS, experiments with AWS ECS :/ Self-hosted K8S on ProfitBricks ooo... K8S can use Azure Active Directory :D
- 45. RUN EVERYWHERE! Get toMicroso ScaleUp, welcome Azure Luckily - Azure Kubernetes Service With GPU integration :) Our OnPrem = Our OnCloud
- 46. KEEP IT SIMPLE azaks CLI for setting k8s - README.rst Terraform for everything else Secrets: 1Password and gopass.pw Terraform also sets our AWS
- 47. MIGRATION WAS EASY... Tech-wiseyes Team-wise no
- 48. DIFFERENCE ☠ Two teamsin Berlin and Warsaw Me in Warsaw
- 49. NEW EXPERIENCE Team overwhelmedwith k8s bits in: TravisCI, Makefiles, Yamls feel it is too much hasle Maybe Keylsey Hightower was right ;)
- 50. APPROACH Simplify, Simplify Hide K8Smagic git tag driven Continoues Deployment
- 51. Repo .travis.yml language: go go: -'1.16' services: - docker install: - curl -sL https://${GITHUB_TOKEN}@raw.githubusercontent.com - if [ -f "tools/travis/install.sh" ]; then bash tools/travi script: - dep ensure - make lint - make test - if [ -z "${TRAVIS_TAG}" ]; then make snapshot; fi; deploy: provider: script # deploy to docker repo
- 52. Makefile Makefile only tasksfor dev |- tools | |- Makefile | |- kube-service.yaml | - kube-deployment.yaml | |- Dockerfile - Makefile
- 53.
- 54. CONTINUOUS DEPLOYMENT 1. gittag and push 2. smacc-platform.git 3. Deploy to staging 4. PR to production
- 55. WHAT WORKED Hiding k8s UnderstandableCD process
- 56. WOULD DO DIFFERENT Moresensitive to feedback
- 57. NEXT Scale our MLtrainings on the top of k8s - kubeflow Use CRD to bring exteranl tools to k8s Keeping an eye on Istio Deployment tool based on missy
- 58. Effective Kubernetes Start withsmall iterations Learn as-you-go Keep Kubernetes Understable Hope the k8s community keep it this way
- 59. Effective Kubernetes Move configurationto runtime Do not terrorize your devs with K8S No free lunch... app must be smarter
- 60. Big thanks tomy colleages in Lyke and SMACC
- 61.
- 62.
- 64.
- 65. INFRA TOOLS Prometheus +AlertMnager + Grafana Traefik Kafka - Yolean/kubernetes-kafka Vault on Etcd - banzaicloud/bank-vaults
- 66. DATA STORES Kafka -Yolean/kubernetes-kafka Etcd - coreos/etcd-operator DB: PSQL and Mongo
- 67. BACKUPS: old-school backups withARK and Restic replications across clouds
- 69. LYKE Now JollyChic Indonesia E-commerce Mobile-only 50k+users 2M downloads Top 10 Fashion Apps w Google Play Store http://www.news.getlyke.com/single- post/2016/12/02/Introducing-the- New-Beautiful-LYKE
- 70. GOOD PARTS Fast Growth A/BTesting Data-driven Product Manager, UI Designer, Mobile Dev, and tester - one body
- 71. 1. CLEAN UP Singlescript for repo - Makefile [1] Resurrect the README [1] With zsh or bash auto-completion plug-in in your terminal.
- 72. 2. GET BACKALL THE KNOWLEDGE Extract from: Puppet, ... ➡ Dockerfile Running Instances ➡ Dockerfile, README.rst Nagios, ... ➡ README.rst + checks/
- 73. 3. INTRODUCE RUN_LOCAL makerun_local A nice section on how to run in README.rst with docker-compose The most crucial point.
- 74. 4. GET TOKUBERNETES make kube_create_config make kube_apply Generate the yaml files if your envs differ secrets from gopass (password manager)
- 75. 5. CONTINUOUS DEPLOYMENT Travis: thesame Makefile as dev use
- 76. 6. KEEP ITRUNNING Bridge the new with old: Use External Services in Kubernetes Expose k8s in the Legacy [1] [1] hard coded IP:PORT, considered: K8S events to Consul
- 77. Bridge the newwith old Service <External> Rabbit Prometheus Exporter RabbitMQ Google AWS Kubernetes EC2 http://rabbitmq:15672 IP 9090 Monitor legacy with new stack
- 78. Architecture During Migration Staging K8S 6 Production K8S 10 GoogleLB Google LB Live 30+ Staging 10+ Google AWS Cloudfront (Live/Staging) api.xstaging-api.x Google LBGoogle LBELB Google LBGoogle LBELB VPN VPN lyke.x staging-lyke.x Prometheus EFK - 2 ElasticSearch 4 nodes Lambdas
- 79. 7. INTRODUCE SMOKE-TEST TARGET_URL=127.0.0make smoke_test TARGET_URL=api.example.com/users make smoke_test
- 80. 8. SERVICE SELF-AWARE Addto old services: 1. metrics/ 2. health/ 3. info/
- 81. 9. MOVE TOMICRO-SERVICES Offload Legacy: Keep the lights on New functionality to micro-services
- 82. WHAT WORKED 1. Copy&PasteMakefile and k8s yaml 2. Separate deployments a good transition strategy
- 83. WHAT DID NOTWORK 1. Too many PoC, cut to 2 weeks max 2. Doing it with too small steps 3. Push back to k8s yaml [*] 4. Alert rules too hard to write [*] With coaching, I thought, it is OK
- 84. DO DIFFERENT 1. Movedata day one 2. Make devs know it is a transition stage 3. Teach earlier about resources 4. EFK could wait 5. All-hands for a paid-XXX% weekend for migration
- 86.
- 87. PODS See each otheron localhost Live and die together Can expose multiple ports Pod nginx WebFiles ENV: HOSTNAME=0.0.0.0 LISTENPORT=8080
- 88.