NIST 800-92 Log Management Guide in the Real World Dr Anton Chuvakin Chief Logging Evangelist

Goals Get a refresher on logs and logging Get familiar with NIST 800-92 Guide Learn how log standards such as NIST help people “in the trenches” Pick a few tips on organizing your log management efforts (if you are a manager) Pick a few logging tips (if you are an analyst)

Get a refresher on logs and logging

Get familiar with NIST 800-92 Guide

Learn how log standards such as NIST help people “in the trenches”

Pick a few tips on organizing your log management efforts (if you are a manager)

Pick a few logging tips (if you are an analyst)

Outline What Logs?  From Log Analysis to Log Management Log Management for Security and Beyond Standards in Logging and Log Management Brief NIST 800-92 Walkthrough How 800-92 Helps You Examples

What Logs? 

From Log Analysis to Log Management

Log Management for Security and Beyond

Standards in Logging and Log Management

Brief NIST 800-92 Walkthrough

How 800-92 Helps You

Examples

Log Data Overview Audit records Transaction logs Intrusion alerts Connection logs System performance records User activity logs Various alerts and other messages Firewalls/intrusion prevention Routers/switches Intrusion detection Servers, desktops, mainframes Business applications Databases Anti-virus VPNs Proxies What Logs? From Where?

Audit records

Transaction logs

Intrusion alerts

Connection logs

System performance records

User activity logs

Various alerts and other messages

Firewalls/intrusion prevention

Routers/switches

Intrusion detection

Servers, desktops, mainframes

Business applications

Databases

Anti-virus

VPNs

Proxies

Security Log Analysis: Why Situational awareness and new threat discovery Unique perspective from combined logs Getting more value out of the network and security infrastructures Get more that you paid for! Measuring security (metrics, trends, etc) Tracking what the users do Incident response (last, but not least!)

Situational awareness and new threat discovery

Unique perspective from combined logs

Getting more value out of the network and security infrastructures

Get more that you paid for!

Measuring security (metrics, trends, etc)

Tracking what the users do

Incident response (last, but not least!)

Log Analysis: Why NOT “ Real hackers don’t get logged !”  Why bother? No, really … Too much data (>x0 GB per day) Too hard to do No tools “that do it for you” Or: tools too expensive What logs? We turned them off 

“ Real hackers don’t get logged !” 

Why bother? No, really …

Too much data (>x0 GB per day)

Too hard to do

No tools “that do it for you”

Or: tools too expensive

What logs? We turned them off 

Log Analysis Basics: How Manual ‘ tail’, ‘more’, etc Filtering Positive and negative (“Artificial ignorance”) Summarization and reports Simple visualization “… worth a thousand words?” Correlation Rule-based and other

Manual

‘ tail’, ‘more’, etc

Filtering

Positive and negative (“Artificial ignorance”)

Summarization and reports

Simple visualization

“… worth a thousand words?”

Correlation

Rule-based and other

From Log Analysis to Log Management

Why Log Management? Logs Beyond Security Threat protection and discovery Regulatory compliance Internal policies and procedure compliance Internal and external audit support Incident response Forensics , “e-discovery” and litigation support IT system and network troubleshooting IT performance management

Threat protection and discovery

Regulatory compliance

Internal policies and procedure compliance

Internal and external audit support

Incident response

Forensics , “e-discovery” and litigation support

IT system and network troubleshooting

IT performance management

From Compliance to Logging Standards Log transmission Syslog (TCP/UDP port 514) Log format Syslog, “a non-standard standard” IDMEF, a failed standard Log contents No standard to speak of: logs = trash can – people dump what they want (or: don’t want!) there Logging practices NIST 800-92 (for security only)

Log transmission

Syslog (TCP/UDP port 514)

Log format

Syslog, “a non-standard standard”

IDMEF, a failed standard

Log contents

No standard to speak of: logs = trash can – people dump what they want (or: don’t want!) there

Logging practices

NIST 800-92 (for security only)

Why Logging Standards? Common language so that people and other systems understand what is in the logs Easier to report on logs and explain the reports Deeper insight into future problems as indicated by the log data Easier system interoperability (thus, reduced cost and complexity) Common logging practices simplify audits and compliance

Common language so that people and other systems understand what is in the logs

Easier to report on logs and explain the reports

Deeper insight into future problems as indicated by the log data

Easier system interoperability (thus, reduced cost and complexity)

Common logging practices simplify audits and compliance

Introducing NIST 800-92 “This publication seeks to assist organizations in understanding the need for sound computer security log management. It provides practical, real-world guidance on developing, implementing, and maintaining effective log management practices throughout an enterprise. “

“This publication seeks to assist organizations in understanding the need for sound computer security log management. It provides practical, real-world guidance on developing, implementing, and maintaining effective log management practices throughout an enterprise. “

NIST 800-92 Walkthrough Introduction to Computer Security Log Management Log Management Infrastructure Log Management Planning Log Management Operational Processes

Introduction to Computer Security Log Management

Log Management Infrastructure

Log Management Planning

Log Management Operational Processes

Computer Security Log Management: Logs “ A log is a record of the events occurring within an organization’s systems and networks” “Within an organization, many logs contain records related to computer security ; common examples of these computer security logs are audit logs that track user authentication attempts and security device logs that record possible attacks.” “This guide addresses only those logs that typically contain computer security-related information .”

“ A log is a record of the events occurring within an organization’s systems and networks”

“Within an organization, many logs contain records related to computer security ; common examples of these computer security logs are audit logs that track user authentication attempts and security device logs that record possible attacks.”

“This guide addresses only those logs that typically contain computer security-related information .”

Computer Security Log Management: Process “ Security log management [is] the process for generating, transmitting, storing, analyzing, and disposing of computer security log data. ”

“ Security log management [is] the process for

generating,

transmitting,

storing,

analyzing, and

disposing

of computer security log data. ”

Computer Security Log Management: Benefits “ It helps to ensure that computer security records are stored in sufficient detail for an appropriate period of time. Routine log reviews and analysis are beneficial for identifying security incidents, policy violations, fraudulent activity, and operational problems […] Logs can also be useful for performing auditing and forensic analysis , supporting the organization’s internal investigations Establishing baselines, and identifying operational trends and long-term problems .”

“ It helps to ensure that computer security records are stored in sufficient detail for an appropriate period of time.

Routine log reviews and analysis are beneficial for identifying security incidents, policy violations, fraudulent activity, and operational problems […]

Logs can also be useful for performing auditing and forensic analysis , supporting the organization’s internal investigations

Establishing baselines, and identifying operational trends and long-term problems .”

Security Logs vs. Security Logs  Logs from Security Applications vs. Security Logs from Applications A key distinction!

Logs from Security Applications

vs.

Security Logs from Applications

A key distinction!

Log Management Challenges “First, there are several potential problems with the initial generation of logs because of their variety and prevalence. Second, the confidentiality, integrity, and availability of generated logs could be breached inadvertently or intentionally. Finally, the people responsible for performing log analysis are often inadequately prepared and supported.”

“First, there are several potential problems with the initial generation of logs because of their variety and prevalence.

Second, the confidentiality, integrity, and availability of generated logs could be breached inadvertently or intentionally.

Finally, the people responsible for performing log analysis are often inadequately prepared and supported.”

Log Management Infrastructure Three Tiers of Log Management Architecture Log Generation Log Analysis and Storage Log Monitoring

Three Tiers of Log Management Architecture

Log Generation

Log Analysis and Storage

Log Monitoring

Log Management Infrastructure: Buzzwords Parsing Filtering Aggregation Rotation Archival Compression Reduction Conversion Normalization Integrity Checking Correlation Viewing Reporting Clearing

Parsing

Filtering

Aggregation

Rotation

Archival

Compression

Reduction

Conversion

Normalization

Integrity Checking

Correlation

Viewing

Reporting

Clearing

Log Management Infrastructure: Tools Syslog-based tools SIEM/SIM/SEM Where did the host IDS go?  Log visualization tools General log management tools (e.g. LogLogic ) Other tools related to logging

Syslog-based tools

SIEM/SIM/SEM

Where did the host IDS go? 

Log visualization tools

General log management tools (e.g. LogLogic )

Other tools related to logging

Log Management Planning: Roles “ Who is invited to the party?” System and network admins Security admins CIRTs Application developers ISOs and CSOs CIOs Auditors And all software buyers

“ Who is invited to the party?”

System and network admins

Security admins

CIRTs

Application developers

ISOs and CSOs

CIOs

Auditors

And all software buyers

Log Management Planning: Policies Policies need to cover “ Log generation Log transmission Log storage and disposal Log analysis“

Policies need to cover

“ Log generation

Log transmission

Log storage and disposal

Log analysis“

Example Policy

Log Management Operational Processes “ Configure the log sources , including log generation, storage, and security Perform analysis of log data Initiate appropriate responses to identified events Manage the long-term storage of log data.”

“ Configure the log sources , including log generation, storage, and security

Perform analysis of log data

Initiate appropriate responses to identified events

Manage the long-term storage of log data.”

Log Security Issues “ Limit access to log files. Avoid recording unneeded sensitive data . Protect archived log files. Secure the processes that generate the log entries. Configure each log source to behave appropriately when logging errors occur. Implement secure mechanisms for transporting log data from the system to the centralized log management servers”

“ Limit access to log files.

Avoid recording unneeded sensitive data .

Protect archived log files.

Secure the processes that generate the log entries.

Configure each log source to behave appropriately when logging errors occur.

Implement secure mechanisms for transporting log data from the system to the centralized log management servers”

Log Analysis Operational Processes Automation is key! Review logs =/= read logs More data is good; context data is better There might be some log entries that you’d never understand  Analyze to prioritize the efforts

Automation is key! Review logs =/= read logs

More data is good; context data is better

There might be some log entries that you’d never understand 

Analyze to prioritize the efforts

Critical Issue: System-level vs. Infrastructure-level Important separation of responsibilities Sysadmin vs. CSO or CIRT Local vs. global Event vs. incident Event response is not the same as incident response Typically, event is system-level while incident infrastructure-level (or organization-level)

Important separation of responsibilities

Sysadmin vs. CSO or CIRT

Local vs. global

Event vs. incident

Event response is not the same as incident response

Typically, event is system-level while incident infrastructure-level (or organization-level)

Manage Long Term Storage A surprisingly hard problem! “ Choose a log format for the data to be archived Archive the log data Verify the integrity of the transferred logs Store the media securely”

A surprisingly hard problem!

“ Choose a log format for the data to be archived

Archive the log data

Verify the integrity of the transferred logs

Store the media securely”

How 800-92 Helps You! Government (under FISMA mandate) Security Manager Security Analyst Commercial Security Manager Security Analyst

Government (under FISMA mandate)

Security Manager

Security Analyst

Commercial

Security Manager

Security Analyst

Government: Manager NIST is voluntary guidance, but FISMA is not (FISMA requires log management): “NIST developed this document in furtherance of its statutory responsibilities under the Federal Information Security Management Act (FISMA) of 2002, Public Law 107-347. “ Planning a log management project? Don’t start from scratch – start from NIST 800-92! Log management touches the whole enterprise, and the guide explains how to involve other teams , not just security

NIST is voluntary guidance, but FISMA is not (FISMA requires log management): “NIST developed this document in furtherance of its statutory responsibilities under the Federal Information Security Management Act (FISMA) of 2002, Public Law 107-347. “

Planning a log management project? Don’t start from scratch – start from NIST 800-92!

Log management touches the whole enterprise, and the guide explains how to involve other teams , not just security

Other: Manager NIST 800-92 might not apply to you directly, but why ignore good advice ? Planning a log management project? Don’t start from scratch – start from NIST 800-92! Compliance drives log management: NIST guide covers a compliance-friendly way of doing log management (and it helps justify management decisions)

NIST 800-92 might not apply to you directly, but why ignore good advice ?

Planning a log management project? Don’t start from scratch – start from NIST 800-92!

Compliance drives log management: NIST guide covers a compliance-friendly way of doing log management (and it helps justify management decisions)

Government and Other: Technical The guide is mostly about process , less bits and bytes … Log collection configuration guidance: how to solve “what to log question” Log analysis tips, including prioritization Storage conundrum : not as simple as sounds What to do about log security ?

The guide is mostly about process , less bits and bytes …

Log collection configuration guidance: how to solve “what to log question”

Log analysis tips, including prioritization

Storage conundrum : not as simple as sounds

What to do about log security ?

Example: NIST 800-92 and PCI Compliance Retail organization log management project driven by PCI DSS Log management in Requirement 10 and beyond NIST guide for tool selection NIST guide for template policies NIST guide for ongoing project success

Retail organization log management project driven by PCI DSS

Log management in Requirement 10 and beyond

NIST guide for tool selection

NIST guide for template policies

NIST guide for ongoing project success

Take These Home with You!! Find the critical systems where logging is essential Enable logging! Read the NIST 800-92 guide (at least the parts needed) – get it on the NIST site http://csrc.nist.gov/publications/nistpubs/ Involve different teams in logging initiatives Look at your logs! You’d be happy you started now and not tomorrow Automate log management

Find the critical systems where logging is essential

Enable logging!

Read the NIST 800-92 guide (at least the parts needed) – get it on the NIST site http://csrc.nist.gov/publications/nistpubs/

Involve different teams in logging initiatives

Look at your logs! You’d be happy you started now and not tomorrow

Automate log management

Thanks for Attending! Dr Anton Chuvakin, GCIA, GCIH, GCFA Chief Logging Evangelist LogLogic, Inc http://www.chuvakin.org See www.info-secure.org for my papers, books, reviews and other security resources Also visit my blog at www.securitywarrior.org

Dr Anton Chuvakin, GCIA, GCIH, GCFA

Chief Logging Evangelist

LogLogic, Inc

http://www.chuvakin.org

See www.info-secure.org for my papers, books, reviews

and other security resources

Also visit my blog at www.securitywarrior.org