Your SlideShare is downloading. ×
0
CSN08: You Can't Correlate
What You Don't Have

Scott Carlson & Rick Yetter
Apollo Group Inc.
Apollo Group

The Apollo Group Challenge
Apollo Group is a publicly traded parent company that
owns the University of Phoe...
3
What’s this about?
• ArcSight products are awesome, as long as you send them
information. The products can’t do anything u...
Our Environment
• 4,500 Servers
– Oracle Enterprise Linux, Red Hat Linux
– Windows 2000, 2003, 2008
– Solaris 9,10

•
•
•
...
It’s the Logs that matter
“The powerful correlation engine of ArcSight ESM sifts through MILLIONS OF LOG
RECORDS to find t...
Where to store the logs
• Long term storage is critical for large companies
– Determine retention requirements (30 days, 1...
Syslog Relay
• Red Hat Linux
• Syslog-ng v4 running on multiple ports
– For receiving logs from multiple sources with uniq...
How to Get the Logs
•
•
•
•
•

Built In Syslog
Syslog-NG
SNARE or Epilog agent, kiwi
File-Reader ArcSight Connector
Someth...
Single Endpoint
• For smaller environments, or environments with fewer layer-2
boundaries
• Should configure server with r...
Single Endpoint with DR Site
• Makes a copy of all logs to an alternate site
• Saves you in case of catastrophic failure
•...
Local Collect & Forward
• Individual Syslog collection in each major network block or
international location

12
How to send the Logs
• Configure syslog
– *.debug @loghost.mydomain.com
– *.*
@loghost

(Solaris)
(Linux)

• Configure SNA...
Decision points
• What’s your Double-Send point?
– Host
• Not available in all “free logging tools”
• Some things cannot d...
Redundancy and Double-Sending
• Fail-over scenarios in use for Apollo Group using Syslog-ng
– Redundancy at the Syslog Rel...
Syslog-NG Configuration
• Syslog-ng configuration (4 simple steps)
– Simple Configuration
• Source
– Where are the logs co...
Syslog-NG Configuration Sample
•
•
•
•
•
•
•

source s_local {
internal();
unix-stream("/dev/log");
file("/proc/kmsg" prog...
Syslog-ng destinations (local)
•
•
•
•
•
•
•
•
•
•
•

#
# Local destinations
#
destination d_messages { file("/u01/log/mes...
Syslog-ng destinations (remote)
•
•
•
•
•
•
•
•
•
•
•
•
•
•

# Remote Destinations
#
destination d_forward { udp("10.3.4.5...
Crazy Use Case #1
Get the unique customer # out of a sub-url string within the debug
log of a firewall in order to perform...
Crazy Use Case #2
Forward non-security events directly to your NOC Console, email
queue, or whatever
• Syslog filter or ex...
Crazy Use Case #3
Gather logs from a proxy server, at 5 minute intervals, and make
sure that they’re going to your DR Site...
Multiple ArcSight ESM Instances
• Double sending all logs allows you have two independent
ArcSight ESM instances, in multi...
Q&A

24
Upcoming SlideShare
Loading in...5
×

You Can't Correlate what you don't have - ArcSight Protect 2011

3,464

Published on

In this presentation we discuss gathering data with syslog-ng in order to properly feed your SIEM system such as ArcSight ESM. This presentation is from HP/ArcSight Protect 2011.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,464
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
50
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Reference: http://www.arcsight.com/collateral/case_studies/ArcSight_CaseStudy_Apollo.pdf
  • Apollo Group Inc. is a public company, founded in 1973.We believed that lifelong employment with a single employer would be replaced by lifelong learning and employment with a variety of employers. Lifelong learning requires an institution dedicated solely to the education of working adults
  • Transcript of "You Can't Correlate what you don't have - ArcSight Protect 2011"

    1. 1. CSN08: You Can't Correlate What You Don't Have Scott Carlson & Rick Yetter Apollo Group Inc.
    2. 2. Apollo Group The Apollo Group Challenge Apollo Group is a publicly traded parent company that owns the University of Phoenix and a number of other subsidiaries in the education arena. With 300 physical locations in six countries, 500,000 students, 50,000 faculty and 22,000 employees, Apollo Group has a formidable challenge in securing all its systems, data and endpoints. Reference: http://www.arcsight.com/collateral/case_studies/ArcSight_CaseStudy_Apollo.pdf 2
    3. 3. 3
    4. 4. What’s this about? • ArcSight products are awesome, as long as you send them information. The products can’t do anything unless you send them as much data as possible! • Normal steps to implementation 1. 2. 3. 4. 5. Define Use Cases Send Logs Build Correlation Rules in ArcSight ESM … Profit!!! 4
    5. 5. Our Environment • 4,500 Servers – Oracle Enterprise Linux, Red Hat Linux – Windows 2000, 2003, 2008 – Solaris 9,10 • • • • • • • 60% Virtualized on VMWARE Multiple international locations & data centers Firewalls (Cisco, Juniper, Checkpoint) Proxy (BlueCoat) IDS (SourceFire) AV/HIPS/DLP (McAfee) …. 5
    6. 6. It’s the Logs that matter “The powerful correlation engine of ArcSight ESM sifts through MILLIONS OF LOG RECORDS to find the critical incidents that matter” (www.arcsight.com) Server Stuff  Security Events  Change Monitoring  Failure Events  Application Logs  Web Logs  Host Firewalls  Active Directory Activity Network Stuff  Firewalls  Proxies  Intrusion Detection  Antivirus  Data Loss Prevention  Email Traffic & Alerts  Wireless  Network Change Monitoring 6
    7. 7. Where to store the logs • Long term storage is critical for large companies – Determine retention requirements (30 days, 1 year, infinite) • Determine who may need the logs, do you need them online? – SysAdmin, Forensics, InfoSec, Legal • Do you need non-repudiation? • Determine Storage method – Splunk – ArcSight Logger - Filesystem(s) full of Raw Log files - Alternate Logging Product • If you build your own – SAN versus NAS versus Local JBOD. You need to log even if things break! 7
    8. 8. Syslog Relay • Red Hat Linux • Syslog-ng v4 running on multiple ports – For receiving logs from multiple sources with unique filters • Local JBOD w/12TB configured as RAID-5 – Make sure you can log even if your SAN is borked! • Additional security of SAMHAIN, tripwire, Solid Core to protect your files from modification • 64GB of ram • Lots of processors 2
    9. 9. How to Get the Logs • • • • • Built In Syslog Syslog-NG SNARE or Epilog agent, kiwi File-Reader ArcSight Connector Something entirely custom, just put it in a FILE! – Syslog format or CEF Format, you pick. You found something without logs???? Well, Ask the developer or company to add logging!!!! 9
    10. 10. Single Endpoint • For smaller environments, or environments with fewer layer-2 boundaries • Should configure server with redundancy in mind in case of failure • Can use file reader connector to read from local logs • Single destination, easy to script • May not scale • Limited to small number of networks unless you traverse firewalls 2
    11. 11. Single Endpoint with DR Site • Makes a copy of all logs to an alternate site • Saves you in case of catastrophic failure • Adds bandwidth to the WAN or remote site link Data Center Alternate Facility Server(s) Relay 2
    12. 12. Local Collect & Forward • Individual Syslog collection in each major network block or international location 12
    13. 13. How to send the Logs • Configure syslog – *.debug @loghost.mydomain.com – *.* @loghost (Solaris) (Linux) • Configure SNARE – Destination Snare Server address – Destination Port – Enable SYSLOG Header = loghost = 514 = Selected • Read the Fine Manual of your product to enable logging with a remote destination. If that’s not there, write to a file! 13
    14. 14. Decision points • What’s your Double-Send point? – Host • Not available in all “free logging tools” • Some things cannot double send (network gear, appliances) – Relay • Adds cross-data center traffic times # Relay – Central • Easy to control flow, exposure is at this point in each DC • Blind to logs if central server is gone • What about things that don’t have syslog? – File Reader to multiple ArcSight ESM Targets is a possibility 14
    15. 15. Redundancy and Double-Sending • Fail-over scenarios in use for Apollo Group using Syslog-ng – Redundancy at the Syslog Relay level • Logs are sent from Snare agents on Windows or by Syslog to relays – Each Syslog relay has a VM hot standby in case of a hardware failure. • Each Syslog relay is configured to send all information received to multiple central servers for redundancy and fault tolerance. • Each Syslog relay retains all logs received for a period of 30 days before being rotated out. 15
    16. 16. Syslog-NG Configuration • Syslog-ng configuration (4 simple steps) – Simple Configuration • Source – Where are the logs coming from? UDP, TCP, File • Destination – Where are you going to send the logs? Disk, output to TCP or UDP? » Can you handle the TCP Overhead? • Filters – Keep what you want, discard the rest! • Log – Log the source, process it, send it to the destination. • Encrypted communications must use TCP 16
    17. 17. Syslog-NG Configuration Sample • • • • • • • source s_local { internal(); unix-stream("/dev/log"); file("/proc/kmsg" program_override("kernel:")); udp(ip(0.0.0.0) port(514) flags(store-legacy-msghdr)); udp(ip(10.11.12.13) port(514) flags(store-legacy-msghdr)); }; • • • • • • • • • • • • • • # # Local filters # filter f_boot { facility(local1); }; filter f_messages { level(info..emerg); }; filter f_secure { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_cron { facility(cron); }; filter f_emerg { level(emerg); }; filter f_spooler { level(crit..emerg) and facility(uucp, news); }; filter f_local7 { facility(local7); }; # Snare *NIX Filters # #filter f_filter_nix { match(":") and not match("snmp") and not match("printd") and not match("-6-302013") and not match("-6-302015") and not match("kernel") and not match("lpstat") and not match("Application") and not match("System") and not host("10.29.10.100") and not match("dhcpd") and not match("xinetd") and not match("puppetmasterd") and not match("crond") and not match("multipathd") and not match("modprobe"); }; 17
    18. 18. Syslog-ng destinations (local) • • • • • • • • • • • # # Local destinations # destination d_messages { file("/u01/log/messages"); }; destination d_secure { file("/u01/log/secure"); }; destination d_maillog { file("/u01/log/maillog"); }; destination d_cron { file("/u01/log/cron"); }; destination d_console { usertty("root"); }; destination d_spooler { file("/u01/log/spooler"); }; destination d_bootlog { file("/u01/log/boot.log"); }; # 18
    19. 19. Syslog-ng destinations (remote) • • • • • • • • • • • • • • # Remote Destinations # destination d_forward { udp("10.3.4.5" port(514) keep_alive(no)); }; # # Local logs # log { source(s_local); filter(f_emerg); destination(d_console); }; log { source(s_local); filter(f_secure); destination(d_secure); }; log { source(s_local); filter(f_mail); destination(d_maillog); flags(final); }; log { source(s_local); filter(f_cron); destination(d_cron); flags(final); }; log { source(s_local); filter(f_spooler); destination(d_spooler); }; log { source(s_local); filter(f_boot); destination(d_bootlog); }; log { source(s_local); destination(d_messages); }; log { source(s_local); destination(d_forward); }; 19
    20. 20. Crazy Use Case #1 Get the unique customer # out of a sub-url string within the debug log of a firewall in order to perform tracking/troubleshooting • Debug Logging ON on firewall (LOTS OF TRAFFIC!) • Logs send to Syslog • Syslog filter or external program called to trim out the customer number and write it to a separate file 20
    21. 21. Crazy Use Case #2 Forward non-security events directly to your NOC Console, email queue, or whatever • Syslog filter or external program called to grab the events you’re interested in, and send them to external mailer (mail –s “alert”) or a syslog-ng filter • Don’t forget the System Administrators • IMHO 90% of problems are misconfigured systems 21
    22. 22. Crazy Use Case #3 Gather logs from a proxy server, at 5 minute intervals, and make sure that they’re going to your DR Site with minimal delay, add a filter to find naughty surfing. • • • • Proxy server sends logs via SCP to syslog relay Syslog relay writes file to local JBOD Syslog-ng or local script scrapes naughtiness from file Cron job runs at 5 minute intervals to SCP completed files to DR Watch out for incomplete files! Make sure your formatting is good!!! 22
    23. 23. Multiple ArcSight ESM Instances • Double sending all logs allows you have two independent ArcSight ESM instances, in multiple data centers capable of performing your SOC duties at a moments notice. 23
    24. 24. Q&A 24
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×