Enterprise Log
Management
Jerod Brennen, CISSP
1

Presentation History
Prepared for:
The Central Ohio InfoSec Forum VIII
Buckeye Hall of Fame Café
May 17, 2006
2

Overview
► The Basics
► Building the Infrastructure
► Battle Plan
► Culture Shock
► Reference Material
3

The Basics
4

But First, a Monkey on a Bicycle…
Image downloaded from http://www.brownandbrown.tv/bat/images/monkey-400.png
http://www.brownandbrown.tv/bat/images/monkey-
5

The Basics
Answer me these
questions three…
Image from Monty Python and the Holy Grail
What am I trying to accomplish?
►
Log Management or Security Event Management?
Who do I need to engage internally?
►
Evaluators, Funding
What are my requirements?
►
General, Unique
6

What am I trying to accomplish?
Log Management
►
Capture all logs, retain them, and extract useful
information from them
Regulatory compliance (long term retention)
Forensics / Investigations (change)
Security Event Management
►
Capture relevant logs, analyze them, and generate alerts
on possible security incidents
Incident response (short term analysis)
Forensics / Investigations (incidents)
Know thy vendor!
7

Who do I need to engage?
► Information Security Team(s)
► Command Center
► System Performance
► Internal Audits
► Who else?
Who will benefit from this information?
Who will pay for this architecture?
8

What are my requirements?
Every company has a different set of
►
requirements. Understanding those
requirements is vital before selecting
vendors to evaluate.
1) Identify scope (input from all parties)
2) Refine scope
3) Define scope (sign-off!)
9

Hardware (Appliances) vs. Software
► Appliances ► Software
Pros Pros
► Easy to manage ► Scalability
is
inexpensive (add
► Easy to support
another server)
Cons
Cons
► Scalability
is expensive
► Multiplegroups involved
(add another appliance)
in management and
support
10

Requirements Broken Down
► Capture
► Store
► Report
► Alert
11

Requirements - Capture
► Identify all production systems
► Identify log formats
Syslog, Windows Event Logs, SMF records
(Mainframe), custom (applications)
► Identify message rate
Events per second
► Identify uncompressed message volume
Increased network traffic
Increased storage
12

Requirements – Storage
► Syslog servers
► Log capture appliances (with local storage)
► Network Area Storage (NAS)
► Content Area Storage (CAS)
How much detail do you really need?
What are your retention requirements?
13

Requirements - Reporting
► Who needs to receive reports?
Sys Admins
Security
Management
► How granular do reports need to be?
14

Requirements - Alerting
► Who needs to receive alerts?
Security
Telecomm
System Performance
► Arethere any formatting or detail
requirements?
15

Architectural Diagrams
► Ask vendors for an architectural
diagram of their solution in your
environment before you schedule any
on-site evaluations.
► Considerations include…
Appliances vs. Servers
Multiple data centers
Multiple possible architectures
►Storage and traffic considerations
16

More Monkeys…
Image downloaded from http://www.cardcow.com/images/set58/thumbs/card00518_fr.jpg
17

Building the
Infrastructure
18

Building the Infrastructure
► Identifylog sources
► Calculate traffic and storage
► Evaluate vendors (preparation)
► Open source components
19

Identify Log Sources
► Asset management system
► System administrators
► Asset flags
Dev / Test / Prod
Regulatory (SOX, HIPPA, PCI)
Sensitive (Personally Identifiable
Information, Privacy Data, Trade Secrets)
20

Calculate Traffic and Storage
► 1st technique: Actuals
Setup a temporary syslog server
Gather during vendor evals
► 2nd technique: Estimate
Traffic calculators
Storage calculators
21

Evaluate Vendors (Preparation)
► Developweighted criteria based on
your requirements
Ask vendors for their templates
► Conductdog-and-pony shows for
evaluators
Ask them to rank vendors based on
relevance
22

Open Source Components
► Whatis your company’s policy on open
source technologies?
Operating Systems (Linux)
Databases (MySQL)
Web Servers (Apache)
Applications
►OSSIM (Open Source Security Information
Management, http://www.ossim.net/)
23

A Monkey Smoking a Cigarette…
Image downloaded from http://www.hatebus.com/locker/smoking_monkey.jpg
http://www.hatebus.com/locker/smoking_monkey.jpg
24

Battle Plan
25

Battle Plan
► Research
► Evaluate
► Implement
Putting the entire process in perspective…
26

Research
► Analysts
Gartner, Forrester, Burton Group
► White papers and webcasts
SANS, log management vendors
► Conferences
SANS Log Management Summit
Catalyst Conference (Burton Group)
Computer Security Institute
27

Evaluate
► Identify vendors of interest
Architecture, Cost
► Schedule on-site demos
High level, multiple groups
Ask participants to rank, based on relevance
Weighted criteria document
► Schedule on-site evaluations
Detailed, based on interest in demos
Document results (test cases, exec summary)
28

Implement
► Selectarchitecture
► Develop deployment plan
List of systems to be monitored
Phased vs. Full
► Purchase infrastructure
29

Monkey Nerd
Image downloaded from http://www.taibros.net/images/monkey-nerd.jpg
http://www.taibros.net/images/monkey-
30

Culture Shock
Image downloaded from http://movies.apple.com/trailers/sony_pictures/spider-man_2/trailer/images/main_image.jpg
http://movies.apple.com/trailers/sony_pictures/spider-
“With great power there must also come --
great responsibility!”
- Stan Lee (or FDR?)
31

Culture Shock
► Who Else Wants to Play?
► The Orwellian Enterprise
32

Who Else Wants to Play?
► Following a successful implementation,
it’s likely that additional groups will
request to use the new tool.
► Be prepared to capture this
information to help demonstrate the
value of security.
33

The Orwellian Enterprise
► Nowthat we have all of this data,
what do we do with it?
Educate all users (management and staff) about
“appropriate use” of the new tool
Inform all employees about the new enterprise
logging architecture
Publish a new (or modify an existing) log
management policy / standard
Strike a balance between effective security and
efficient operations
34

Reference
Material
35

Additional Resources
► Log management conference
► Foundational reading
► Volume and storage calculators
► Toolkit
► Vendors
► Documents to ask for
36

Log Management Conference
► SANS Log Management Summit
July 12-14, Washington D.C.
http://www.sans.org/logmgtsummit06/
37

Foundational Reading
NIST Guide to Computer Security Log Management
►
http://csrc.ncsl.nist.gov/publications/drafts/DRAFT-SP800-92.pdf
http://csrc.ncsl.nist.gov/publications/drafts/DRAFT- SP800-
The Log Management Industry: An Untapped Market (SANS Analyst Program)
►
http://www.sans.org/vendor/20050426_analyst_report.pdf
Managing Security Event Information: Compliance Is a Propelling Force
►
(Burton Group; client login required)
http://www.burtongroup.com/research_consulting/doc.aspx?cid=280
Evaluating and Deploying Security Information and Event Management
►
Technologies (Gartner; client login required)
http://www.gartner.com/DisplayDocument?doc_cd=129132
http://www.gartner.com/DisplayDocument?doc_cd=129132
SANS “What Works in Log Management”
►
Webcast, http://www.sans.org/webcasts/show.php?webcastid=90597
http://www.sans.org/webcasts/show.php?webcastid=90597
Document from LogLogic, http://www.loglogic.com/documents/case-studies/citizens-bank-case-
http://www.loglogic.com/documents/case- studies/citizens- bank- case-
study.pdf
38

Log Analysis Links
► Log Analysis mailing list
http://lists.shmoo.com/mailman/listinfo/loganalysis
► Log Analysis website
Links to related tools and articles
http://www.loganalysis.org/
39

Volume and Storage Calculators
► Message volume
Messages per second (mps) / Events per second (eps)
Depends on the level of detail in the logs (debug level)
Requires some analysis of your environment
► Storage requirements
Bytes per event * Events per second * Number of devices
Factor in compression (varies by vendor)
Estimated average event size
► 1,000 bytes per event for Windows servers
► 250 bytes per event for UNIX servers
► 150 bytes per event for firewalls and other telecom devices
40

Toolkit
SNARE (System iNtrusion Analysis and Reporting Environment)
►
http://www.intersectalliance.com/projects/index.html
“… a program that provides a central collection facility for a variety of log sources, including Snare
variety
Agents for Windows, Solaris, AIX, Irix, ISA Server, IIS Server, Apache, Lotus Notes (and others),
Irix,
plus any device capable of sending data to a syslog server.”
server.”
Project Lasso (agentless Windows event log gathering)
(agentless
►
http://www.loglogic.com/logforge/
“… provide centralized log management, eliminating the need to manage individual agents, and,
manage
greatly reducing the impact on monitored servers in terms of storage and processing.”
storage processing.”
SNARE Generator (testing tool)
►
http://www.intersectalliance.com/projects/index.html
http://www.intersectalliance.com/projects/index.html
“… a(n) Event Simulator used by InterSect Alliance as part of our normal Snare Server testing
a(n)
process.”
process.”
DBAN (Darik’s Boot and Nuke)
(Darik’
►
http://dban.sourceforge.net/
“… automatically and completely delete the contents of any hard disk that it can detect, which
disk
makes it an appropriate utility for bulk or emergency data destruction.”
destruction.”
41

Vendors
Appliances Software
► ►
Cisco ArcSight
LogLogic Computer Associates
Network Intelligence Consul
Symantec eIQ Networks
TriGeo e-Security
GuardedNet
IBM Tivoli
Intellitactics
Micromuse
netForensics
NetIQ
OpenService
SenSage
42

Documents to ask for
Magic Quadrant for Security Information and
►
Event Management (Gartner)
Don’t limit yourself by quadrant! Consider each vendor
based on your requirements.
The Forrester Wave: Security Information
►
Management (Forrester)
This is the summary document. Evaluated vendors will
have documents specific to their products.
Case studies (successful implementations)
►
43

A Monkey Doing My Job…
Image downloaded from http://img.engadget.com/common/images/6561335245723223.JPG?0.3590002168255687
http://img.engadget.com/common/images/6561335245723223.JPG?0.3590002168255687
44

One Final Note
The PowerPoint version of this presentation is 445kb.
By stripping out all of the unnecessary images,
regardless of how funny monkeys are, this file
would only be 140kb.
That’s 68% less required storage, achieved by
analyzing the content and determining what is
essential in terms of retention.
Keep that in mind…
45

Contact Info
Request copies of presentation, questions:
Jerod Brennen
jerod_brennen@yahoo.com
http://www.linkedin.com/jerodbrennen
46

Any questions?
47