Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Code Review goes Cloud Native - Gerrit on Kubernetes managed by Gardener

285 views

Published on

Gerrit User Summit 2018 Palo Alto on Nov 16, 2018
https://gerrit.googlesource.com/summit/2018/+/master/index.md

Published in: Software
  • Be the first to comment

  • Be the first to like this

Code Review goes Cloud Native - Gerrit on Kubernetes managed by Gardener

  1. 1. Code Review goes Cloud Native Gerrit on Kubernetes managed by Gardener
  2. 2. Gerrit Master MySQL Master Fetch GerritMaster gitgc Push Git Repositories read write gc
  3. 3. Add Slaves MySQL Master Fetch GerritMaster gitgc Push Git Repositories read write gc
  4. 4. Add Slaves MySQL Master Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gc
  5. 5. Add Slaves MySQL Master Apache-Git- backend Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gc
  6. 6. Add Slaves MySQL Master Apache-Git- backend Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread
  7. 7. Add Slaves MySQL Master Apache-Git- backend Git Repositories write Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  8. 8. Add Slaves MySQL Master Apache-Git- backend https: git push Git Repositories write Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  9. 9. Add Slaves MySQL Master Apache-Git- backend MySQL Slave https: git push Git Repositories write Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  10. 10. Add Slaves MySQL Master Apache-Git- backend MySQL Slave https: git push fetch transaction log Git Repositories write Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  11. 11. Add Slaves MySQL Master Apache-Git- backend GerritSlave MySQL Slave https: git push fetch transaction log Git Repositories write Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  12. 12. Add Slaves MySQL Master Apache-Git- backend GerritSlave MySQL Slave https: git push fetch transaction log Git Repositories write read Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  13. 13. Add Slaves MySQL Master Apache-Git- backend GerritSlave MySQL Slave https: git push fetch transaction log Git Repositories write read Fetch Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  14. 14. Add Slaves MySQL Master Apache-Git- backend gitgc GerritSlave MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  15. 15. Add Slaves MySQL Master Apache-Git- backend gitgc GerritSlave MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  16. 16. Add Slaves MySQL Master Apache-Git- backend gitgc GerritSlave MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread ssh: create repos
  17. 17. Dockerize Slaves MySQL Master Apache-Git- backend gitgc GerritSlave MySQL Slave ssh: create repos https: git push fetch transaction log Git Repositories write gc read Fetch Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread
  18. 18. From Containers to Kubernetes
  19. 19. From Containers to Kubernetes VM Host OS Container Runtime Container
  20. 20. From Containers to Kubernetes VM Host OS Container Runtime Container
  21. 21. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Container
  22. 22. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container
  23. 23. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
  24. 24. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
  25. 25. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
  26. 26. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
  27. 27. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Container Scheduler Container
  28. 28. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Declarative – not procedural • Declare target state, reconcile to desired state • Self-healing Container Scheduler Container
  29. 29. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Docker Workload Portability • Abstract from cloud provider specifics • Multiple container runtimes Declarative – not procedural • Declare target state, reconcile to desired state • Self-healing Container Scheduler Container
  30. 30. Gerrit-Slave Helm Chart MySQL Helm Chart PV Slaves on Kubernetes MySQL Master Kubernetes deployment Apache-Git CronJob git gc Persistent Volume Git Repositories Fetch Fetch GerritMaster gitgc Replication Plugin Push Git Repositories read write gcread deployment replica set Gerrit Slave NFS Helm Chart NFS Server MySQL Slave Storage ingress ingress PVC PVC PVC PVC
  31. 31. Gerrit-Slave Helm Chart MySQL Helm Chart PV Kubernetes Cluster deployment Apache-Git CronJob git gc Persistent Volume Git Repositories Fetch deployment replica set Gerrit Slave MySQL Slave NFS Volume ingress ingress PVC PVC PVC PVC Gerrit-Master Helm Chart MySQL Helm Chart PV job git gc Persistent Volume Git Repositories deployment replica set Gerrit Master MySQL Master NFS Volume PVC PVC PVC Fetch Push Master-Slave on Kubernetes
  32. 32. Gardener - Kubernetes Clusters as a Service Gardener
  33. 33. Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  34. 34. Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  35. 35. Worker Seed Cluster Master Master Master Worker Worker HA Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  36. 36. Worker Seed Cluster Master Master Master Worker Worker HA Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  37. 37. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  38. 38. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster GardenerapiVersion: garden.sapcloud.io/v1 kind: Shoot metadata: name: my-cluster namespace: garden-project spec: dns: provider: aws-route53 domain: cluster.ondemand.com cloud: aws: networks: vpc: cidr: 10.250.0.0/16 workers: - name: cpu-worker machineType: m4.xlarge autoScalerMin: 5 autoScalerMax: 20 kubernetes: version: 1.11.2 kubeAPIServer: featureGates: ... runtimeConfig: ... admissionPlugins: ... maintenance: timeWindow: begin: 220000+0000 end: 230000+0000 autoUpdate: kubernetesVersion: true status: ... cluster.yaml
  39. 39. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  40. 40. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  41. 41. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  42. 42. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Gardener
  43. 43. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service WorkerMaster Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker Gardener Zooming into the Seed Cluster reveals…
  44. 44. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes WorkerMaster Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server SchedulerController Mgr ETCD Gardener Zooming into the Seed Cluster reveals…
  45. 45. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes WorkerMaster Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API ServerETCD SchedulerController Mgr API Server SchedulerController Mgr ETCD Gardener Zooming into the Seed Cluster reveals…
  46. 46. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes WorkerMaster Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API ServerETCD SchedulerController Mgr API Server SchedulerController Mgr ETCD ETCD …API Server Gardener Zooming into the Seed Cluster reveals…
  47. 47. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener Machine Controller Manager Machine Provisioning Self-Healing Auto-Update Auto-Scaling Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes WorkerMaster Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API ServerETCD SchedulerController Mgr API Server SchedulerController Mgr ETCD ETCD …API Server Gardener Zooming into the Seed Cluster reveals…
  48. 48. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener Machine Controller Manager Machine Provisioning Self-Healing Auto-Update Auto-Scaling Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes WorkerMaster Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API ServerETCD SchedulerController Mgr API Server SchedulerController Mgr ETCD ETCD …API Server Gardener Zooming into the Seed Cluster reveals…
  49. 49. Demo Gerrit on Kubernetes managed by Gardener
  50. 50. Next steps Storage • use Ceph / Rook to replace NFS Update chart to enable using master branch • get rid of database
  51. 51. Index Replace Lucene by ElasticSearch
  52. 52. Gerrit Operator based on operator framework to implement • Sharding • Multi-master • Auto-Scaling
  53. 53. Logging EFK-Stack • Elastic search => Indexing • Fluentbit => Log-collection • Kibana => Visualization
  54. 54. Monitoring Prometheus => data collection Grafana => visualization
  55. 55. Contributions welcome ! Picture Link
  56. 56. Links Git Repository https://gerrit.googlesource.com/k8s-gerrit Docker Images https://hub.docker.com/u/k8sgerrit Gardener https://gardener.cloud

×