Download to read offline
Uploaded byCodeOps Technologies LLP
Running Elasticsearch {Stack} on Kubernetes
Elasticsearch is the world's most popular open source search engine. Kubernetes (k8s) is the popular container orchestration engine giving developers the flexibility to run all sorts of workloads easily. Elastic launched the Elasticsearch k8s operator sometime ago. With this, one can not only run Elasticsearch on k8s but also can launch other Elastic Stack solutions like APM Server. Help run rolling upgrades, manage data etc. Presented as part of Cloud Community Days on 17th June - ccdays.konfhub.com
![Lect 1 Number systems and base conversions. [Autosaved].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/lect1numbersystemsandbaseconversions-260111134109-67c2d865-thumbnail.jpg?width=640&height=640&fit=bounds)
Running Elasticsearch {Stack} on Kubernetes
- 1. Running Elasticsearch {Stack}on Kubernetes @aravindputrevu
- 2. @aravindputrevu Hello Aravind Putrevu Developer Advocate aravind.dev| @aravindputrevu
- 3.
- 4.
- 5.
- 6.
- 7.
- 8. @aravindputrevu Btw, what isKubernetes?
- 9. @aravindputrevu Kubernetes resources ina nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes Node Node Node Pod container container Pod container Pod container Pod container container container Pod container
- 10. @aravindputrevu Kubernetes resources ina nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes ○ Access (TCP) your apps using Services (round-robin on healthy Pods) Node Node Node Pod container container Pod container Pod container Pod container container container Pod container Service Service
- 11. @aravindputrevu Kubernetes resources ina nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes ○ Access (TCP) your apps using Services (round-robin on healthy Pods) ○ Mount Volumes and Secrets into Pods Node Node Node Pod container Pod container Pod container Pod container container Pod container Service Service Volume (local) Secret
- 12. @aravindputrevu Kubernetes resources ina nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes ○ Access (TCP) your apps using Services (round-robin on healthy Pods) ○ Mount Volumes and Secrets into Pods Node Node Node Pod container container Pod container Pod container Pod container container container Pod container Service Service Volume (NAS) Volume (local) Secret
- 13. @aravindputrevu Kubernetes resources ina nutshell • Many more resources exist: ○ Deployment: N replicas of the same Pod ○ DaemonSet: same Pod running on all Nodes ○ PersistentVolume: a Volume that can be reattached to different Pods ○ StatefulSet: N replicas of the same Pod, with PersistentVolumes automatically reattached to a given Pod ○ etc.
- 14. @aravindputrevu Kubernetes resources ina nutshell • Resources are yaml (or json) objects with a well-defined schema
- 15. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node NodeNode kubelet kubelet kubelet Controllers Controllers Controllers controllers user
- 16. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node NodeNode kubelet kubelet kubelet Controllers Controllers Controllers controllers user etcd ● Persistent distributed key-value store, organized as a filesystem ● With a watch mechanism ● Stores all k8s resources
- 17. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node NodeNode kubelet kubelet kubelet Controllers Controllers Controllers controllers user apiserver ● API to create/update/delete k8s resources ● Handles authentication & authorization ● Horizontally scalable ● With a watch mechanism
- 18. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node NodeNode kubelet kubelet kubelet Controllers Controllers Controllers controllers user user ● Interacts with apiserver HTTP API ● Often with kubectl (CLI)
- 19. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node NodeNode kubelet kubelet kubelet Controllers Controllers Controllers controllers user kubelet ● Agent running on each Node ● Watches Pods in the apiserver ● Manages corresponding containers on the host
- 20. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node NodeNode kubelet kubelet kubelet Controllers Controllers Controllers controllers user controllers ● Watch resources in the apiserver ● Process them with their own logic ○ May createanew resources on the apiserver ○ May interact with external systems
- 21. @aravindputrevu Kubernetes architecture • Someexisting controllers: ○ scheduler: watches Pods, assigns them to a Node ○ replication-controller: maintains the correct number of Pods as defined in a ReplicaSet (-> Deployment) ○ endpoints-controller: joins Pods IP/port in Services ○ cloud-provider-X-controller: create eg. EBS volumes ○ etc.
- 22.
- 23.
- 24. 2424 Package Manager forKubernetes github.com/elastic/helm-charts
- 25.
- 26.
- 27.
- 28.
- 29. @aravindputrevu ● Custom ResourcesDefinitions (CRD) ○ Elasticsearch, Kibana, APM ● A set of controllers ● Automate Tasks ○ Generate TLS certificates ○ Create Secrets holding configuration & certs ○ Create a Service to access the cluster ○ Create ES users ○ Deployment Templates Elastic Cloud on Kubernetes
- 30.
- 31.
- 32.