Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deploy Prometheus - Grafana and EFK stack on Kubic k8s Clusters

Deploy Prometheus - Grafana and EFK stack on Kubic k8s Clusters

  • Be the first to comment

  • Be the first to like this

Deploy Prometheus - Grafana and EFK stack on Kubic k8s Clusters

  1. 1. Deploy Monitoring and Centralized Logging with Prometheus + Grafana and EFK Stack on Kubic k8s clusters Syah Dwi Prihatmoko / sdmoko sdmoko@opensuse.org
  2. 2. Syah Dwi Prihatmoko Moko @sdmoko
  3. 3. openSUSE-ID Member GNU Linux Bogor Activist Cloud Engineer at Btech FOSS Contributor and Enthusiast
  4. 4. In this talks, we'll learn how to deploy monitoring and logging with Prometheus - Grafana and EFK Stack in Kubic k8s clusters. The example is to monitoring and logging pods in k8s clusters.
  5. 5. Kubic?
  6. 6. Deploy k8s on Kubic openSUSE MicroOS Kubic came with kubeadm to make it easy to bootstrapping a kubernetes cluster. Just run this command to init a k8s cluster. kubeadm init --pod-network-cidr=10.244.0.0/16 After this you need to configure user to talk to cluster by running mkdir -p ~/.kube cp -i /etc/kubernetes/admin.conf ~/.kube/config
  7. 7. Deploy k8s on Kubic The next step is to set up the network plugin. Kubic recommended to use flannel. Run the command to set up flannel. kubectl apply -f /usr/share/k8s-yaml/flannel/kube-flannel.yaml After this you can join the worker node, to get the join command execute this command on master node. kubeadm token create --print-join-command https://blog.sdmoko.net/create-k8s-cluster-with-kubic.html
  8. 8. Monitoring Vs Logging
  9. 9. EFK Stack Elasticsearch, Fluentd and Kibana
  10. 10. EFK Deployment Guide can be found on my blog. https://blog.sdmoko.net/efk-cluster -on-kubernetes.html
  11. 11. Deploy EFK Stack The YAML file available at https://github.com/sdmoko/EFK-kubernetes First create a storage class kubectl apply -f 00-storageClass.yaml After this we need to create PV. kubectl apply -f 01-pv-0.yaml kubectl apply -f 02-pv-1.yaml kubectl apply -f 03-pv-2.yaml
  12. 12. Deploy EFK Stack Then we need to create a namespaces for a logging system. kubectl apply -f 04-namespaces.yaml After this create a service for elasticsearch cluster, so the Fluentd and Kibana can use it. kubectl apply -f 05-elasticsearch_svc.yaml Then create a cluster elasticsearch which use the volume that we create before. kubectl apply -f 06-elasticsearch_statefulset.yaml
  13. 13. Deploy EFK Stack Then deploy Kibana and Fluentd to make it a complete logging stack. Where we can visualize the data. kubectl apply -f 07-kibana.yaml kubectl apply -f 08-fluentd.yaml
  14. 14. Kibana Dashboard
  15. 15. Prometheus + Grafana
  16. 16. Prometheus + Grafana
  17. 17. Prometheus + Grafana deployment guide can be found on my blog. https://blog.sdmoko.net/deploy-prometheus-gr afana-on-kubernetes.html
  18. 18. Deploy Prometheus + Grafana The yaml for this deployment can be found here https://github.com/sdmoko/k8s-prometheus-grafana First, we need to deploy prometheus as a metric storage (TSDB). To deploy prometheus we need to add a RBAC for prometheus. kubectl apply -f prometheus/00-prometheus-rbac.yaml Then deploy a configmap for prometheus config. kubectl apply -f prometheus/01-prometheus-configmap.yaml
  19. 19. Deploy Prometheus + Grafana Then we can create a rules that will trigger the alertmanager. kubectl apply -f prometheus/02-prometheus-rules.yaml Then we can create a storage for prometheus store the data. kubectl apply -f prometheus/03-prometheus-storage.yaml Then we can create a deployment and service for prometheus. kubectl apply -f prometheus/prometheus-deployment.yaml kubectl apply -f prometheus/prometheus-service.yaml
  20. 20. Deploy Prometheus + Grafana Then we deploy kube-metrics server to expose container and pod metrics other than those exposed by cadvisor on the nodes. kubectl apply -f kube-state-metrics/kube-state-metrics.yaml Then we can deploy and provide service for grafana dashboard to visualize the monitoring data. kubectl apply -f grafana/grafana-deployment.yaml kubectl apply -f grafana/grafana-service.yaml
  21. 21. Deploy Prometheus + Grafana Then we open the grafana service from browser, and then add prometheus data source and add a dashboard. For config of the grafana data source use: Name : DS_Prometheus Type : Prometheus URL : http://prometheus-service:8080
  22. 22. Grafana Dashboard
  23. 23. Questions ?
  24. 24. Reference https://github.com/sdmoko/EFK-kubernetes https://github.com/sdmoko/k8s-prometheus-grafana https://medium.com/faun/production-grade-kubernetes-monitoring-using-prometheus-78144b835b60 https://www.digitalocean.com/community/tutorials/how-to-set-up-an-elasticsearch-fluentd-and-kibana-efk-logging-stack-o n-kubernetes
  25. 25. Terima Kasih. Thank You.

×