Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker & Elastic Stack

3,798 views

Published on

Prashansa Kulshrestha is currently pursuing B. Tech in Computer Science with Specialization in Cloud Computing and Virtualization Technology at UPES, Dehradun. She is Red Hat OpenShift Certified Engineer as well as RHCE too. She has experience working on Docker, Docker Compose & Docker Swarm. She is an active contributor of Docker Labs. She is expert in Python and well acquainted with AWS.

She recently implemented Log analytics tool using Elastic Stack and Docker and want to share her learning and experience with Collabnix Community.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker & Elastic Stack

  1. 1. Integrated Log Analytics And Monitoring Of Servers using Docker & Elastic Stack. Date: 21st November 2019
  2. 2. Internal Use - Confidential Who Am I? Prashansa Kulshrestha  Red Hat Certified Specialist in OpenShift Administration  Red Hat Certified Specialist in Ansible Automation  Red Hat Certified Engineer | System Admin  Pursuing B.Tech (UPES)  LinkedIn: https://linkedin.com/in/prashansa-k  GitHub: https://github.com/Prashansa-K
  3. 3. Internal Use - Confidential Agenda  Introduction to Docker  Introduction to Elastic Stack  Need for Log Analysis  About Project  Infrastructure Management using Redfish API  Project Approach  Workflow  Results
  4. 4. Internal Use - Confidential Introduction To Docker Docker: A tool that allows to create containers. Why to use Docker?  Standardized packaging for software and its dependencies to achieve platform- neutrality.  Isolating applications from each other.  Fast way to deploy applications.  Portable.  Lightweight.  Supported on many platforms.
  5. 5. Internal Use - Confidential Elastic Stack
  6. 6. Internal Use - Confidential Elastic Stack
  7. 7. Internal Use - Confidential Elasticsearch  Uses JSON format for data storage.  Uses REST API for communication.  Uses Inverted Indexing for Fast Retrieval of data. RDBMS Elasticsearch Database Index Tables Types Records/Rows Documents Attributes/ Columns Properties Name College Prashansa UPES { “Name”: “Prashansa”, “College”: “UPES” }
  8. 8. Internal Use - Confidential Elasticsearch Indexing in RDBMS Indexing in Elasticsearch
  9. 9. Internal Use - Confidential Elastic Stack
  10. 10. Internal Use - Confidential Logstash  Used for collecting, parsing and filtering log data.  Can be used to create data pipelines. input { file { path => "/var/log/apache/access.log" type => "apache" } } filter { } output { elasticsearch { hosts => ["localhost:9200"] } }
  11. 11. Internal Use - Confidential Elastic Stack
  12. 12. Internal Use - Confidential Kibana  Visualization tool.  Has a Web-based UI.  Can be used to create visual representations like graphs, pie-charts, etc.
  13. 13. Internal Use - Confidential Need for Log Analysis  Debugging  Predictive Analysis  Unstructured Logs  Security Analysis  Performance Analysis
  14. 14. Internal Use - Confidential About the Project Aim To collect server logs in an Elasticsearch cluster and analyze them for errors or warnings. Why Docker and ELK? Project's focus was to accumulate hardware-level logs such as CPU, Memory, I/O utilizations, power fluctations, sensor logs, etc. Such logs are not accessible at OS-level and can't be sent to servers like syslog directly.  Elasticsearch was used as a log store.  Docker containers were used to fetch the logs using Redfish API.
  15. 15. Internal Use - Confidential Redfish API  RESTful API specification  Not vendor-specific.  Clients send URI requests over https to iDRAC, so clients can be any OS or application on a server, workstation or mobile device.
  16. 16. Internal Use - Confidential Approach  One container per Server Model.  Tracking machines using their iDRAC IP Addresses.  Collecting logs in every 10 minutes and pushing them to Elasticsearch database.  Logs being segregated based on the Message IDs or Error Codes which help to analyze the current condition and working of the system under consideration.  Generating alerts incase of undesirable situations.  Automated Workflow
  17. 17. Internal Use - Confidential Project Architecture
  18. 18. Developing A Recommendation Tool For Optimizing Operational Workloads For Servers
  19. 19. Internal Use - Confidential Log Accumulation & Analysis Logs get accumulated in Elasticsearch every 10 minutes. For every server, we have a separate dedicated index which stores its lifecycle logs. Naming of the index is done as such: For a server with IDRAC IP as 100.98.26.xx, index would be named as index100.98.26.xx When the script runs for the first time, Volume V logs get collected in the index. And then in every 10 minutes, we get some ∆Vn logs over the old logs. Thus, our indices keep building up.
  20. 20. Internal Use - Confidential Analysis Workflow  The Search Algorithm would be fed with the newly generated logs and it would find the Message ID which is frequently generated.  A priority scheme will be used to segregate the Message IDs and finding the source of problem.  Most frequently occurring log message with the highest priority will be treated as the main cause of problem.  This can be fed to our recommendation system which can suggest what steps to take next to resolve the error and prevent the system from crashing.
  21. 21. Search Algorithm Working
  22. 22. Internal Use - Confidential Results: Periodic Log Accumulation
  23. 23. Internal Use - Confidential Results: Periodic Log Accumulation
  24. 24. Internal Use - Confidential Results: Kibana Visualizations
  25. 25. Internal Use - Confidential Results: Kibana Visualizations
  26. 26. Results: Slack Alerts
  27. 27. Results: Slack Alerts
  28. 28. Thank You

×