Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Log Management
An Introduction
Lennart Koopmann
Jordan Sissel
What is a Log?
time + data
What is a Log?
37.5.55.31 - - [08/Apr/2014:15:31:30 -0400]
"GET /images/web/2009/banner.png HTTP/1.1"
304 - "http://semicomplete.com/styl...
120707 0:40:34 4 Connect root@localhost on
4 Query select @@version_comment limit 1
120707 0:40:45 4 Query select * from m...
Kinds of Logs
Trace and Debug
Kinds of Logs
Accounting
Kinds of Logs
Transaction
Kinds of Logs
Problems
Difficult to Access
Problems
Too Many Logs
Problems Difficult to Access
Too Many Servers
Problems Difficult to Access
No Permissions :(
Problems Difficult to Access
Difficult to Consume
Problems
Unstructured
Problems Difficult to Consume
Requires Expertise
Problems Difficult to Consume
Requires Maintenance
Problems
Configuration
Problems Requires Maintenance
Log Retention
Problems Requires Maintenance
Bad Tooling
Problems
grep, ssh, awk
Problems Bad Tooling
Life of a Log
Life of a Log
Record
Transport
Search &
Analyze
Archive
Delete
Sources of Logs
Vendor Hardware
Sources of Logs
Routers, VPNs, Printers, Phones,
AWS CloudTrail, etc
Vendor Software
Sources of Logs
Nginx, Wordpress, Jira
In-house Software
Sources of Logs
Your company controls it
Solutions
(Open Source!)
Solutions
Logstash
from Elasticsearch
Graylog2
from Torch
Upcoming SlideShare
Loading in …5
×

OSDC 2014: Jordan Sissel & Lennart Koopmann - Intro to log management

507 views

Published on

Log management can be overwhelming! This talk provide an overview of log management. You will learn about different kinds of logs, structured/unstructured logs, retention, archival, and more. We will also briefly introduce several common tools in the log management space.

Published in: Software
  • Be the first to comment

OSDC 2014: Jordan Sissel & Lennart Koopmann - Intro to log management

  1. 1. Log Management An Introduction Lennart Koopmann Jordan Sissel
  2. 2. What is a Log?
  3. 3. time + data What is a Log?
  4. 4. 37.5.55.31 - - [08/Apr/2014:15:31:30 -0400] "GET /images/web/2009/banner.png HTTP/1.1" 304 - "http://semicomplete.com/style2.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0" What is a Log? time + data
  5. 5. 120707 0:40:34 4 Connect root@localhost on 4 Query select @@version_comment limit 1 120707 0:40:45 4 Query select * from mysql.user What is a Log? time + data ?????? ???????
  6. 6. Kinds of Logs
  7. 7. Trace and Debug Kinds of Logs
  8. 8. Accounting Kinds of Logs
  9. 9. Transaction Kinds of Logs
  10. 10. Problems
  11. 11. Difficult to Access Problems
  12. 12. Too Many Logs Problems Difficult to Access
  13. 13. Too Many Servers Problems Difficult to Access
  14. 14. No Permissions :( Problems Difficult to Access
  15. 15. Difficult to Consume Problems
  16. 16. Unstructured Problems Difficult to Consume
  17. 17. Requires Expertise Problems Difficult to Consume
  18. 18. Requires Maintenance Problems
  19. 19. Configuration Problems Requires Maintenance
  20. 20. Log Retention Problems Requires Maintenance
  21. 21. Bad Tooling Problems
  22. 22. grep, ssh, awk Problems Bad Tooling
  23. 23. Life of a Log
  24. 24. Life of a Log Record Transport Search & Analyze Archive Delete
  25. 25. Sources of Logs
  26. 26. Vendor Hardware Sources of Logs Routers, VPNs, Printers, Phones, AWS CloudTrail, etc
  27. 27. Vendor Software Sources of Logs Nginx, Wordpress, Jira
  28. 28. In-house Software Sources of Logs Your company controls it
  29. 29. Solutions (Open Source!)
  30. 30. Solutions Logstash from Elasticsearch Graylog2 from Torch

×