Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Shipping & Visualize Your Data With ELK

Introduce Elasticsearch, Logstash and Kibana how to improve log collection.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Shipping & Visualize Your Data With ELK

  1. 1. Shipping & Visualize Your Data With ELK Adam Chen
  2. 2. HELLO Everyone! I am Adam I am just an engineer like to share some experience with others. Thanks Gentoo let me familiar with Linux. You can find me at @adaam
  3. 3. Let’s Start This Session
  4. 4. ELK? What !?
  5. 5. Develop by
  6. 6. Elastic Family
  7. 7. ELK ? DevOps ?
  8. 8. Log Still Play An Important Role
  9. 9. Today’s characters Logstash/Beat series Shipping all of your log to where it should go, like ES, AWS, or just text. Elasticsearch The main part to store your data with High Availability. Kibana Visualize will power your data. To know more about its value.
  10. 10. Traditional Way to Collect Log When error happened, administrator or RD/QA will need to login or write/use tool to grab log from each machines then analysis.
  11. 11. Hey Bob, Please collect the error log to analyze. OK, Boss. BOSS
  12. 12. Hey Bob, Please collect the error log to analyze. Traditional Way to Collect Log
  13. 13. Old Way to Collect Log
  14. 14. How ELK Help ?
  15. 15. Centralize Log To One Place
  16. 16. Collect Log using ELK
  17. 17. Introduce The E, The L and The K
  18. 18. Logstash An agent install at where log need to be collect. Have much filter to process your log. Also Input/Output module. Logstash Module
  19. 19. Logstash Filters Date, geoip, json, kv ...etc GROK !! Grok online tester
  20. 20. Logstash DEMO Input apache/nginx log then output to stdout, using local logstash.
  21. 21. Elasticsearch SImply a data store with near real time search Store data in index, can by hours, day, week, month. Setting at Logstash. Machine spec depends on data size.
  22. 22. Elasticsearch Modules They are improve ES function if you need. Watcher, Shield, Marvel, Cloud-AWS
  23. 23. Elasticsearch DEMO API Webpage
  24. 24. Kibana Show your data to you understand. But you need to know your data fields first.
  25. 25. Kibana Demo Create some template to show (data pregen at ES? ) Real time insert data to ES and see from template pregen. (apache/nginx?)
  26. 26. What else ?
  27. 27. Push Metrics to elsewhere
  28. 28. THANKS! Any questions? You can find me at: @adaam

×