Baselining Logs How to create baselines and analyze logs effectively?

Outline What is a baseline? What is a log? Why baseline? Requirements for log “baselining” Baseline lifecycle What baselines well? What baselines poorly? Examples and how to do it

What is a baseline? What is a log?

Why baseline?

Requirements for log “baselining”

Baseline lifecycle

What baselines well?

What baselines poorly?

Examples and how to do it

Definitions Log = record from a file about computer activities Also: alert, event, alarm, etc Baseline = “A starting point or condition against which future changes are measured”

Log = record from a file about computer activities

Also: alert, event, alarm, etc

Baseline = “A starting point or condition against which future changes are measured”

Log Analysis Methods Manual ‘ Tail’, ‘more’, etc Filtering Positive and negative (“Artificial ignorance”) Summarization and reports Simple visualization “… worth a thousand words?” Simple automation Filters Correlation Rule-based and other methods

Manual

‘ Tail’, ‘more’, etc

Filtering

Positive and negative (“Artificial ignorance”)

Summarization and reports

Simple visualization

“… worth a thousand words?”

Simple automation

Filters

Correlation

Rule-based and other methods

Why Baseline? Situational awareness What is going on compared to some baseline New threat discovery Unique perspective unavailable from other methods Getting more value out of the network and security infrastructures Leverage the stuff you have in new ways Extracting what is really actionable automatically Out of baseline, unusual = bad? Measuring security (metrics, trends, etc) Compliance and regulations

Situational awareness

What is going on compared to some baseline

New threat discovery

Unique perspective unavailable from other methods

Getting more value out of the network and security infrastructures

Leverage the stuff you have in new ways

Extracting what is really actionable automatically

Out of baseline, unusual = bad?

Measuring security (metrics, trends, etc)

Compliance and regulations

Simple Examples Hits on port 80 over the last week User logins to server X per day Use of su command per hour of day Count of new ports hit on a firewall Number of hosts touching each server per hour

Hits on port 80 over the last week

User logins to server X per day

Use of su command per hour of day

Count of new ports hit on a firewall

Number of hosts touching each server per hour

What is needed? Data – and lots of it!  Normalized format across data sources Expert feedback into what is normal and bad Not needed : “training data”!

Data – and lots of it! 

Normalized format across data sources

Expert feedback into what is normal and bad

Not needed : “training data”!

Baseline Assumptions There is data available Past was not disastrous! Baseline is a correct model for the situation at hand won’t work for erratic/random phenomena or will cause “bad baselines”

There is data available

Past was not disastrous!

Baseline is a correct model for the situation at hand

won’t work for erratic/random phenomena or will cause “bad baselines”

Baseline Lifecycle II Create Update Age Compare and act on results Refine

Create

Update

Age

Compare and act on results

Refine

Baseline Lifecycle II

Baseline Creation Pick parameters to baseline E.g. NIDS alerts per sensor Pick a time period and time bin E.g . compare today to last week Pick comparison method E.g. compare today’s count to average

Pick parameters to baseline

E.g. NIDS alerts per sensor

Pick a time period and time bin

E.g . compare today to last week

Pick comparison method

E.g. compare today’s count to average

Compare to Baseline NEW OVER UNDER GONE Newly appeared, over baseline, under baseline (a lot vs a little), disappeared

NEW

OVER

UNDER

GONE

“Interestingness” Something interesting ? One research paper defines “interesting” thus: Unexpected to user Actionable (we can and/or should do something about it) Examples : Compromised/infected system Successful attack Insider abuse and IP theft Covert channel/hidden backdoor communication Increase in probing System crash

Something interesting ?

One research paper defines “interesting” thus:

Unexpected to user

Actionable (we can and/or should do something about it)

Examples :

Compromised/infected system

Successful attack

Insider abuse and IP theft

Covert channel/hidden backdoor communication

Increase in probing

System crash

What Baselines Well? Where different = interesting! New attack type Larger number of bytes Sharp drop in log event flow New usernames More destinations hit

Where different = interesting!

New attack type

Larger number of bytes

Sharp drop in log event flow

New usernames

More destinations hit

Example 1: Can you Guess What Happened?! This visual for this example is censored. The picture would show a one-dimensional of hits to a specific port. Destination Port 1D Baseline

This visual for this example is censored. The picture would show a one-dimensional of hits to a specific port.

Example 2: Can you Guess What Happened?!

Example 4: Can you Guess What Happened?!

Good Baselines [Operationally Tested] Log message type per sensor per day Log message type per protocol/port Log message types (watch for NEW) Protocols per sensor per day Count (unique (alert)) per source Count (unique (port)) per source

Log message type per sensor per day

Log message type per protocol/port

Log message types (watch for NEW)

Protocols per sensor per day

Count (unique (alert)) per source

Count (unique (port)) per source

What Baselines Poorly? Random things Hits on port TCP 3445 anybody?  Things that go up and down for on their own Accesses to a document on a server Sometimes, only large deviations matter

Random things

Hits on port TCP 3445 anybody? 

Things that go up and down for on their own

Accesses to a document on a server

Sometimes, only large deviations matter

Examples

How YOU can do it? First, collect events AANVAL OSSIM OSSEC (?) ACID/BASE Syslog2SQL Whatever SQL log and event store

First, collect events

AANVAL

OSSIM

OSSEC (?)

ACID/BASE

Syslog2SQL

Whatever SQL log and event store

How YOU can do it? Second, plan what to baseline Third, run the tools Fourth, act on the results Mitigate, block, disable, slice-n-dice  Fifth, automate as needed

Second, plan what to baseline

Third, run the tools

Fourth, act on the results

Mitigate, block, disable, slice-n-dice 

Fifth, automate as needed

Summary Easy and effective way to deal with logs from multiple sources Allow to automate log monitoring To some extent Result may be given to less skilled people for follow-up

Easy and effective way to deal with logs from multiple sources

Allow to automate log monitoring

To some extent

Result may be given to less skilled people for follow-up

Q&A? More information? Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA anton@chuvakin.org Security Strategist Author of “Security Warrior” (O’Reilly 2004) – www.securitywarrior.com Book on logs is coming soon! See www.info-secure.org for my papers, books, reviews and other security resources related to logs

Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA

anton@chuvakin.org

Security Strategist

Author of “Security Warrior” (O’Reilly 2004) – www.securitywarrior.com

Book on logs is coming soon!

See www.info-secure.org for my papers, books, reviews and other security resources related to logs