Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Running Elasticsearch {Stack} on Kubernetes @aravindputrevu
@aravindputrevu Hello Aravind Putrevu Developer Advocate aravind.dev | @aravindputrevu
@aravindputrevu Key Takeaways {on}
@aravindputrevu Elastic Stack
@aravindputrevu Elastic Stack
@aravindputrevu Elastic Stack used by...
@aravindputrevu Deploying Elastic Stack
@aravindputrevu Btw, what is Kubernetes?
@aravindputrevu Kubernetes resources in a nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes...
@aravindputrevu Kubernetes resources in a nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes...
@aravindputrevu Kubernetes resources in a nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes...
@aravindputrevu Kubernetes resources in a nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes...
@aravindputrevu Kubernetes resources in a nutshell • Many more resources exist: ○ Deployment: N replicas of the same Pod ○...
@aravindputrevu Kubernetes resources in a nutshell • Resources are yaml (or json) objects with a well-defined schema
@aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controll...
@aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controll...
@aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controll...
@aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controll...
@aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controll...
@aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controll...
@aravindputrevu Kubernetes architecture • Some existing controllers: ○ scheduler: watches Pods, assigns them to a Node ○ r...
@aravindputrevu Deploying Applications in {k8s}
@aravindputrevu Deploying Applications in {k8s}
2424 Package Manager for Kubernetes github.com/elastic/helm-charts
2525 Deploying Applications in {k8s}
@aravindputrevu Helm vs Operator
@aravindputrevu Helm vs Operator
2828 github.com/elastic/cloud-on-k8s
@aravindputrevu ● Custom Resources Definitions (CRD) ○ Elasticsearch, Kibana, APM ● A set of controllers ● Automate Tasks ○...
3030
@aravindputrevu community.elastic.co ela.st/eck-k8s
Thank you Questions? @aravindputrevu
Upcoming SlideShare
Loading in …5
×

Running Elasticsearch {Stack} on Kubernetes

1,286 views

Published on

Elasticsearch is the world's most popular open source search engine. Kubernetes (k8s) is the popular container orchestration engine giving developers the flexibility to run all sorts of workloads easily. Elastic launched the Elasticsearch k8s operator sometime ago. With this, one can not only run Elasticsearch on k8s but also can launch other Elastic Stack solutions like APM Server. Help run rolling upgrades, manage data etc.

Presented as part of Cloud Community Days on 17th June - ccdays.konfhub.com

Published in: Software
no profile picture user

  • Be the first to comment

  • Be the first to like this

Running Elasticsearch {Stack} on Kubernetes

  1. 1. Running Elasticsearch {Stack} on Kubernetes @aravindputrevu
  2. 2. @aravindputrevu Hello Aravind Putrevu Developer Advocate aravind.dev | @aravindputrevu
  3. 3. @aravindputrevu Key Takeaways {on}
  4. 4. @aravindputrevu Elastic Stack
  5. 5. @aravindputrevu Elastic Stack
  6. 6. @aravindputrevu Elastic Stack used by...
  7. 7. @aravindputrevu Deploying Elastic Stack
  8. 8. @aravindputrevu Btw, what is Kubernetes?
  9. 9. @aravindputrevu Kubernetes resources in a nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes Node Node Node Pod container container Pod container Pod container Pod container container container Pod container
  10. 10. @aravindputrevu Kubernetes resources in a nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes ○ Access (TCP) your apps using Services (round-robin on healthy Pods) Node Node Node Pod container container Pod container Pod container Pod container container container Pod container Service Service
  11. 11. @aravindputrevu Kubernetes resources in a nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes ○ Access (TCP) your apps using Services (round-robin on healthy Pods) ○ Mount Volumes and Secrets into Pods Node Node Node Pod container Pod container Pod container Pod container container Pod container Service Service Volume (local) Secret
  12. 12. @aravindputrevu Kubernetes resources in a nutshell • Basically: ○ Run Pods (group of Docker Containers) on a fleet of Nodes ○ Access (TCP) your apps using Services (round-robin on healthy Pods) ○ Mount Volumes and Secrets into Pods Node Node Node Pod container container Pod container Pod container Pod container container container Pod container Service Service Volume (NAS) Volume (local) Secret
  13. 13. @aravindputrevu Kubernetes resources in a nutshell • Many more resources exist: ○ Deployment: N replicas of the same Pod ○ DaemonSet: same Pod running on all Nodes ○ PersistentVolume: a Volume that can be reattached to different Pods ○ StatefulSet: N replicas of the same Pod, with PersistentVolumes automatically reattached to a given Pod ○ etc.
  14. 14. @aravindputrevu Kubernetes resources in a nutshell • Resources are yaml (or json) objects with a well-defined schema
  15. 15. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controllers Controllers Controllers controllers user
  16. 16. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controllers Controllers Controllers controllers user etcd ● Persistent distributed key-value store, organized as a filesystem ● With a watch mechanism ● Stores all k8s resources
  17. 17. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controllers Controllers Controllers controllers user apiserver ● API to create/update/delete k8s resources ● Handles authentication & authorization ● Horizontally scalable ● With a watch mechanism
  18. 18. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controllers Controllers Controllers controllers user user ● Interacts with apiserver HTTP API ● Often with kubectl (CLI)
  19. 19. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controllers Controllers Controllers controllers user kubelet ● Agent running on each Node ● Watches Pods in the apiserver ● Manages corresponding containers on the host
  20. 20. @aravindputrevu Kubernetes architecture etcd apiserver apiserver apiserver Node Node Node kubelet kubelet kubelet Controllers Controllers Controllers controllers user controllers ● Watch resources in the apiserver ● Process them with their own logic ○ May createanew resources on the apiserver ○ May interact with external systems
  21. 21. @aravindputrevu Kubernetes architecture • Some existing controllers: ○ scheduler: watches Pods, assigns them to a Node ○ replication-controller: maintains the correct number of Pods as defined in a ReplicaSet (-> Deployment) ○ endpoints-controller: joins Pods IP/port in Services ○ cloud-provider-X-controller: create eg. EBS volumes ○ etc.
  22. 22. @aravindputrevu Deploying Applications in {k8s}
  23. 23. @aravindputrevu Deploying Applications in {k8s}
  24. 24. 2424 Package Manager for Kubernetes github.com/elastic/helm-charts
  25. 25. 2525 Deploying Applications in {k8s}
  26. 26. @aravindputrevu Helm vs Operator
  27. 27. @aravindputrevu Helm vs Operator
  28. 28. 2828 github.com/elastic/cloud-on-k8s
  29. 29. @aravindputrevu ● Custom Resources Definitions (CRD) ○ Elasticsearch, Kibana, APM ● A set of controllers ● Automate Tasks ○ Generate TLS certificates ○ Create Secrets holding configuration & certs ○ Create a Service to access the cluster ○ Create ES users ○ Deployment Templates Elastic Cloud on Kubernetes
  30. 30. 3030
  31. 31. @aravindputrevu community.elastic.co ela.st/eck-k8s
  32. 32. Thank you Questions? @aravindputrevu

×