Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
LogLogic Confidential Thursday, March 19, 20151
The Top Five Log
Analysis Mistakes
Dr Anton Chuvakin
Chief Logging Evangel...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
2Confidential |
Summary
1. System, Network and Security L...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
3Confidential |
Log Data Overview
 Audit logs
 Transact...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
4Confidential |
Login? Logon? Log in?
<122> Mar 4 09:23:1...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
5Confidential |
“Arrgh! Why
Don’t We Just
Ignore’Em?”
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
6Confidential |
Log Management Mandate and Regulations
Re...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
7Confidential |
So, How Do People Do It?
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
8Confidential |
Log Analysis Basics
 Manual
– ‘Tail’, ‘m...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
9Confidential |
From Log Analysis to Log Management
 Thr...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
10Confidential |
Looks Complicated?! No
Wonder People Mak...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
11Confidential |
Six Mistakes of Log Analysis and Log Man...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
12Confidential |
Mistake 0: Not Logging AT ALL …
… and it...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
13Confidential |
Example: Oracle
 Defaults:
– minimum sy...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
14Confidential |
Mistake 1: Not looking at logs
 Collect...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
15Confidential |
Example Log Review Priorities
1. DMZ NID...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
16Confidential |
Mistake 2: Storing Logs For Too Short A ...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
17Confidential |
Also A Mistake: Storing Logs for TOO LON...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
18Confidential |
Example Retention Strategy
Type + networ...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
19Confidential |
Mistake 3: Deciding What’s Relevant Befo...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
20Confidential |
Example Common Logging Order
Log everyth...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
21Confidential |
Mistake 4: Ignoring Logs from Applicatio...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
22Confidential |
Example: Jumbled Mess of Application Log...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
23Confidential |
Mistake 5: Looking for only the bad stuf...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
24Confidential |
Conclusions: Mistakes Summary
0. Not log...
Thursday, March 19, 2015
Mitigating Risk. Automating Compliance.
25Confidential |
Thanks for Attending the Presentation
Dr...
Upcoming SlideShare
Loading in …5
×

O'Reilly Webinar Five Mistakes Log Analysis

2,691 views

Published on

O'Reilly Webinar on "Five Mistakes Log Analysis"

Published in: Technology, Business
  • Be the first to comment

O'Reilly Webinar Five Mistakes Log Analysis

  1. 1. LogLogic Confidential Thursday, March 19, 20151 The Top Five Log Analysis Mistakes Dr Anton Chuvakin Chief Logging Evangelist LogLogic, Inc
  2. 2. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 2Confidential | Summary 1. System, Network and Security Logs 2. Why Look at Logs? 3. Brief Log Analysis Overview 4. Log Analysis Mistakes
  3. 3. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 3Confidential | Log Data Overview  Audit logs  Transaction logs  Intrusion logs  Connection logs  System performance records  User activity logs  Various alerts and other messages  Firewalls/intrusion prevention  Routers/switches  Intrusion detection  Servers, desktops, mainframes  Business applications  Databases  Anti-virus  VPNs What logs? From Where?
  4. 4. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 4Confidential | Login? Logon? Log in? <122> Mar 4 09:23:15 localhost sshd[27577]: Accepted password for kyle from ::ffff:192.168.138.35 port 2895 ssh2 <13> Fri Mar 17 14:29:38 2006 680 Security SYSTEM User Success Audit ENTERPRISE Account Logon Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 Logon acco unt: POWERUSER Source Workstation: ENTERPRISE Error Code: 0xC000006A 4574 <57> Dec 25 00:04:32:%SEC_LOGIN-5-LOGIN_SUCCESS:Login Success [user:yellowdog] [Source:10.4.2.11] [localport:23] at 20:55:40 UTC Fri Feb 28 2006 <18> Dec 17 15:45:57 10.14.93.7 ns5xp: NetScreen device_id=ns5xp system- warning-00515: Admin User netscreen has logged on via Telnet from 10.14.98.55:39073 (2002-12-17 15:50:53)
  5. 5. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 5Confidential | “Arrgh! Why Don’t We Just Ignore’Em?”
  6. 6. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 6Confidential | Log Management Mandate and Regulations Regulations Require LMI  SOX  GLBA  FISMA  JPA  NIST 800-53  Capture audit records  Regularly review audit records for unusual activity and violations  Automatically process audit records  Protect audit information from unauthorized deletion  Retain audit logs  PCI  HIPAA  SLAs Mandates Demand It  PCI : Requirement 10 and beyond  Logging and user activities tracking are critical  Automate and secure audit trails for event reconstruction  Review logs daily  Retain audit trail history for at least one year  COBIT  ISO  ITIL  COBIT 4  Provide audit trail for root-cause analysis  Use logging to detect unusual or abnormal activities  Regularly review access, privileges, changes  Verify backup completion  ISO17799  Maintain audit logs for system access and use, changes, faults, corrections, capacity demands  Review the results of monitoring activities regularly and ensure the accuracy of logs Controls Require it “Get fined, Get Sanctioned” “Lose Customers, Reputation, Revenue or Job” “Get fined, Go To Jail”
  7. 7. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 7Confidential | So, How Do People Do It?
  8. 8. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 8Confidential | Log Analysis Basics  Manual – ‘Tail’, ‘more’, ‘grep’, ‘notepad’, etc  Filtering – Positive and negative (“Artificial ignorance”)  Summarization and reports – “Top X of Y”  Visualization  Log indexing and searching  Correlation – Rule-based and other  Log data mining
  9. 9. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 9Confidential | From Log Analysis to Log Management  Threat protection and discovery  Incident response  Forensics, “e-discovery” and litigation support  Regulatory compliance  Internal policies and procedure compliance  Internal and external audit support  IT system and network troubleshooting  IT performance management
  10. 10. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 10Confidential | Looks Complicated?! No Wonder People Make Mistakes …
  11. 11. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 11Confidential | Six Mistakes of Log Analysis and Log Management 0. Not logging at all. 1. Not looking at the logs 2. Storing logs for too short a time 3. Prioritizing the log records before collection 4. Ignoring the logs from applications 5. Only looking for “known bad” stuff
  12. 12. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 12Confidential | Mistake 0: Not Logging AT ALL … … and its aggravated version: “… and not knowing that you don’t”  No logging? -> well, no logs for incident response, audits, compliance Got logs? If your answer is ‘NO”, don’t listen further: run and enable logging right now!
  13. 13. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 13Confidential | Example: Oracle  Defaults: – minimum system logging – minimum database server access – no data access logging  So, where is … – data access audit – schema and data change audit – configuration change audit
  14. 14. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 14Confidential | Mistake 1: Not looking at logs  Collection of logs has value!  But review boosts the value 10-fold (numbers are estimates )  More in-depth analysis boosts it a lot more!  Two choices here … – Review after an incident – Ongoing review
  15. 15. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 15Confidential | Example Log Review Priorities 1. DMZ NIDS 2. DMZ firewall 3. DMZ servers with applications 4. Critical internal servers 5. Other servers 6. Select critical application 7. Other applications
  16. 16. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 16Confidential | Mistake 2: Storing Logs For Too Short A Time  You are saying you HAD logs? And how is it useful?  Retention question is a hard one. Truly, nobody has the answer! – Seven years? A year? 90 days? A week? Until the disk runs out?  Common: 90 days online and up to 1-3 years “nearline” or offline
  17. 17. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 17Confidential | Also A Mistake: Storing Logs for TOO LONG?!  Retention = storage + destruction  Why DESTROY LOGS? – Privacy regulations (mostly EU) – Litigation risk management – System resource utilization
  18. 18. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 18Confidential | Example Retention Strategy Type + network + storage tier  IDS + DMZ + online = 90 days  Firewall + DMZ + online = 30 days  Servers + internal + online = 90 days  ALL + DMZ + archive = 3 years  Critical + internal + archive = 5 years  OTHER + internal + archive = 1 year
  19. 19. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 19Confidential | Mistake 3: Deciding What’s Relevant Before Collection  How would you know what is … – … Security-relevant – … Compliance-relevant – … or will solve the problem you’d have TOMORROW!?  Also affects “forensic quality” of logs  Prioritization Challenge – Got ESP?   “Simple” – just grab everything!
  20. 20. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 20Confidential | Example Common Logging Order Log everything Retain most everything Analyze enough Summarize and report on a subset Look at some Act in real-time on a few
  21. 21. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 21Confidential | Mistake 4: Ignoring Logs from Applications  Firewall – Yes, Linux – Yes, Windows – Yes, NIDS and NIPS – Yes but …  Oracle - ?  SAP - ?  Your Application X – No Log standards are coming: MITRE CEE!
  22. 22. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 22Confidential | Example: Jumbled Mess of Application Logs |22:01:40|BTC| 7|000|DDIC | |LC2|Systemerror when executing external command DB6_DATA_COLLECTOR on gneisenau () |22:02:32|BTC| 7|000|DDIC | |R49|Communication error, CPIC return code 020, SAP return code 456 |22:02:32|BTC| 7|000|DDIC | |R5A|> Conversation ID: 38910614 |22:02:32|BTC| 7|000|DDIC | |R64|> CPI-C function: CMSEND(SAP)
  23. 23. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 23Confidential | Mistake 5: Looking for only the bad stuff  Correlation, filters, regex matching – oh, no!   Why such approaches? – You have to know what you are looking for!  Can we somehow just “see what we need to see”? – Data mining technology can help
  24. 24. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 24Confidential | Conclusions: Mistakes Summary 0. Not logging at all. 1. Not looking at the logs 2. Storing logs for too short a time 3. Prioritizing the log records before collection 4. Ignoring the logs from applications 5. Only looking for “known bad” stuff
  25. 25. Thursday, March 19, 2015 Mitigating Risk. Automating Compliance. 25Confidential | Thanks for Attending the Presentation Dr Anton Chuvakin, GCIH, GCFA Chief Logging Evangelist http://www.chuvakin.org Coauthor of “Security Warrior” (O’Reilly, 2004) and “PCI Compliance” (Syngress, 2007) See http://www.info-secure.org for my papers, books, reviews and other security resources related to logs. Book on logs is coming soon! Also see http://chuvakin.blogspot.com

×