Six Mistakes of Log Management Dr Anton Chuvakin, GCIA, GCIH, GCFA Six Mistakes of Log Management

Summary The World of System, Network and Security Logs Why Look at Logs? Brief Log Analysis Overview From Log Analysis to Log Management Log Management Mistakes: from 0 to 5 Conclusions

The World of System, Network and Security Logs

Why Look at Logs?

Brief Log Analysis Overview

From Log Analysis to Log Management

Log Management Mistakes: from 0 to 5

Conclusions

Log Data Overview Audit records Transaction logs Intrusion alerts Connection logs System performance records User activity logs Various alerts and other messages Firewalls/NIPS Routers/switches Intrusion detection Servers, desktops, mainframes Business applications Databases Anti-virus VPNs What logs? From Where?

Audit records

Transaction logs

Intrusion alerts

Connection logs

System performance records

User activity logs

Various alerts and other messages

Firewalls/NIPS

Routers/switches

Intrusion detection

Servers, desktops, mainframes

Business applications

Databases

Anti-virus

VPNs

What Commonly “Gets Logged”? System or software startup, shutdown, restart, and abnormal termination (crash) Various thresholds being exceeded or reaching dangerous levels such as disk space full, memory exhausted, or processor load too high Hardware health messages that the system can troubleshoot or at least detect and log Access to resources and authentication decisions Network connections , failed and successful User access privilege changes such as the su command—both failed and successful User credentials and access right changes , such as account updates, creation, and deletion—both failed and successful System configuration changes and software updates—both failed and successful

System or software startup, shutdown, restart, and abnormal termination (crash)

Various thresholds being exceeded or reaching dangerous levels such as disk space full, memory exhausted, or processor load too high

Hardware health messages that the system can troubleshoot or at least detect and log

Access to resources and authentication decisions

Network connections , failed and successful

User access privilege changes such as the su command—both failed and successful

User credentials and access right changes , such as account updates, creation, and deletion—both failed and successful

System configuration changes and software updates—both failed and successful

“ Arrgh! Why Don’t We Just Ignore’Em?”

“ Arrgh! Why Don’t We Just Ignore’Em?”

Regulations Mandate Logging and Log Review ISO 17799 Maintain audit logs for system access and use, changes, faults, corrections, capacity demands Review the results of monitoring activities regularly Ensure the accuracy of the logs NIST 800-53 Capture audit records Regularly review audit records for unusual activity and violations Automatically process audit records Protect audit information from unauthorized deletion Retain audit logs PCI Requirement 10, etc Logging and user activities tracking are critical Automate and secure audit trails for event reconstruction Review logs daily Retain audit trail history for at least one year COBIT Provide adequate audit trail for root-cause analysis Use logging and monitoring to detect unusual or abnormal activities Regularly review access, privileges, changes Monitor performance Verify backup completion and NIST 800-92 “Guide to Security Log Management!”

ISO 17799

Maintain audit logs for system access and use, changes, faults, corrections, capacity demands

Review the results of monitoring activities regularly

Ensure the accuracy of the logs

NIST 800-53

Capture audit records

Regularly review audit records for unusual activity and violations

Automatically process audit records

Protect audit information from unauthorized deletion

Retain audit logs

PCI

Requirement 10, etc

Logging and user activities tracking are critical

Automate and secure audit trails for event reconstruction

Review logs daily

Retain audit trail history for at least one year

COBIT

Provide adequate audit trail for root-cause analysis

Use logging and monitoring to detect unusual or abnormal activities

Regularly review access, privileges, changes

Monitor performance

Verify backup completion

NIST 800-92 “Guide to Computer Security Log Management” The first ever official guidance on solving logging challenges Logging configurations Logging policies and procedures Log analysis tools and resources

The first ever official guidance on solving logging challenges

Logging configurations

Logging policies and procedures

Log analysis tools and resources

So, How Do People Do It?

So, How Do People Do It?

Log Analysis Basics Manual ‘ Tail’, ‘more’, ‘grep’, ‘notepad’, etc Filtering Positive and negative (“Artificial ignorance”) Summarization and reports “ Top X of Y” Simple visualization “… worth a thousand words?” Correlation Rule-based and other Log data mining

Manual

‘ Tail’, ‘more’, ‘grep’, ‘notepad’, etc

Filtering

Positive and negative (“Artificial ignorance”)

Summarization and reports

“ Top X of Y”

Simple visualization

“… worth a thousand words?”

Correlation

Rule-based and other

Log data mining

Looks Complicated?! No Wonder People Make Mistakes …

Looks Complicated?! No Wonder People Make Mistakes …

Six Mistakes of Log Management 0. Not logging at all. 1. Not looking at the logs 2. Storing logs for too short a time 3. Prioritizing the log records before collection 4. Ignoring the logs from applications 5. Only looking at what you know is bad

0. Not logging at all.

1. Not looking at the logs

2. Storing logs for too short a time

3. Prioritizing the log records before collection

4. Ignoring the logs from applications

5. Only looking at what you know is bad

Conclusions Now you know: What are the logs? Where they come from? Why look at them? How people do it? What are some of the relevant regulations? How to deal with them? And how to AVOID MISTAKES in log management !

Now you know:

What are the logs?

Where they come from?

Why look at them?

How people do it?

What are some of the relevant regulations?

How to deal with them?

And how to AVOID MISTAKES in log management !

Thanks for Attending!!! Dr Anton Chuvakin, GCIA, GCIH, GCFA Chief Logging Evangelist http://www.chuvakin.org Author of “Security Warrior” (O’Reilly, 2004) – http://www.securitywarrior.org See http://www.info-secure.org for my papers, books, reviews and other security resources related to logs. Book on logs is coming soon!

Dr Anton Chuvakin, GCIA, GCIH, GCFA

Chief Logging Evangelist

http://www.chuvakin.org

Author of “Security Warrior” (O’Reilly, 2004) – http://www.securitywarrior.org

See http://www.info-secure.org for my papers, books, reviews and other security resources related to logs. Book on logs is coming soon!