Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker logging with Elastic stack

858 views

Published on

The number of containers running in production is growing. A container zoo can be very hard to debug and analyze. In this demonstration, let's explore how to collect logs/metrics from Docker Host and Docker Containers with the help of Elastic Beats, ElasticSearch and Kibana.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Docker logging with Elastic stack

  1. 1. 01
  2. 2. 02
  3. 3. Let's start! 03
  4. 4. Dockerrrr 04
  5. 5. Java 05
  6. 6. Docker and Java 06
  7. 7. Docker Flow 07
  8. 8. Ship artifacts 08
  9. 9. Container packaging 09
  10. 10. Container packaging 10
  11. 11. Container packaging 11
  12. 12. Yeah! 12
  13. 13. Ouch! 13
  14. 14. Logging 14
  15. 15. ELK ((L)LEK) 15
  16. 16. ELK ((L)LEK) 16
  17. 17. ELK (B(L)EK) 17
  18. 18. ELK (B(L)EK) 18
  19. 19. ELK (BEK) 19
  20. 20. Docker logs Show container's STDERR and STDOUT. docker logs [‐f] [‐t] CONTAINER_ID01. 20
  21. 21. Logging driver $ docker info | grep 'Logging Driver' Logging Driver: json‐file 01. 02. 21
  22. 22. Find the logs $ docker run ‐itd jenkins 67aa... $ tail ‐f /var/lib/docker/containers/67aa.../67aa..._json.log ... {"log":"INFO: Jenkins is fully up and runningrn",  "stream":"stdout","time":"2017‐02‐06T02:15:23.087459653Z"} 01. 02. 03. 04. 05. 06. 22
  23. 23. Redirecting to syslog sudo docker run ‐i ‐t ‐d     ‐‐log‐driver=syslog     ‐‐log‐opt syslog‐address=tcp://127.0.0.1:5000     jenkins 01. 02. 03. 04. 23
  24. 24. Volumes If application has several log files, they can be shared through volumes. Logging agent can monitor those volumes and send data to log aggregators. • • 24
  25. 25. Let's play with it Start ElasticSearch in a container. Change PetClinic to log in JSON format. Start File Beat in a container and make it send PetClinic container JSON log to ElasticSearch. Start Kibana in a container and observe the result. • • • • 25
  26. 26. Hints Use https://github.com/logstash/logstash­logback­encoder.• 26
  27. 27. Hints docker network create elk docker run ‐dit ‐‐net=elk ‐‐name=elasticsearch ‐p 9200:9200 ‐p 9300:9300 elasticsearch:5.2.0 docker run ‐dit ‐‐net=elk ‐‐name=kibana ‐p 5601:5601 kibana:5.2.0 • • • 27
  28. 28. Hints (filebeat.yml) filebeat.prospectors: ‐ input_type: log     paths: ["...."]     json.keys_under_root: true     json.overwrite_keys: true     json.add_error_key: true     json.message_key: "message"     tags: ["json"] 01. 02. 03. 04. 05. 06. 07. 08. 28
  29. 29. Hints (filebeat.yml) output.elasticsearch:   hosts: ["elasticsearch:9200"] 01. 02. 29
  30. 30. Conclusion 30
  31. 31. That's all! 31
  32. 32. Thank you! 32
  33. 33. 33
  34. 34. Upcoming trainings 34
  35. 35. DevOps MasterClass 35
  36. 36. Training: DevOps MasterClass 36
  37. 37. Training: DevOps MasterClass http://bit.ly/devops_stockholm Stockholm, 27th­28th of March, 22nd­23rd of May, 2017 http://bit.ly/devops_riga Riga, 8th­9th of May, 2017 • • • • 37
  38. 38. JVM Internals 38
  39. 39. Training: JVM Internals 39
  40. 40. Training: JVM Internals http://bit.ly/jvm_stockholm Stockholm, 24th­25th of May, 2017 http://devchampions.com/training/jvm/ Riga, 18th­19th of May, 2017 • • • • 40

×