Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Centralized Logging with syslog

8,753 views

Published on

Published in: Technology, Education

Centralized Logging with syslog

  1. 1. Building Centralized Logging: Syslog Steven “Maniac” McGrath
  2. 2. Syslog? • logging service • UNIX based • Networkable
  3. 3. Wait a Sec...Network? • UDP port 514 • Typically limited to 1024bytes
  4. 4. One more thing... • FIFO Buffers • First In First Out • Rolling View of Logs • Type of Named Pipe
  5. 5. FIFO...Tasty *chomp* 3 Line FIFO Buffer Item 5 Item 4 Item 3 Item 2 Item 1
  6. 6. Getting Started... • Ubuntu 6.06 Server • Base Install
  7. 7. Installing Syslog... • Update The Repository
  8. 8. Upgrade the OS • We need to upgrade the OS to current.
  9. 9. Install Syslog-NG • Syslog-NG will remove klogd, this is normal.
  10. 10. Reconfiguring Syslog-ng • Configuration depends on network environment. • Windows Hosts • Cisco Devices • Linux Hosts • Other Devices and Gear
  11. 11. First off...Global! /etc/syslog-ng/syslog-ng.conf options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); group(admin); perm(0640); dir_perm(0755); use_dns(no); stats_freq(0); }; • Disable Hostname Chaining • Time to wait before re-establishing a dead connection • Time to wait before an idle file is closed • FIFO Buffer size • Create Directories • Permissions • Disable DNS • Disable Statistic Logging
  12. 12. Next, The Source /etc/syslog-ng/syslog-ng.conf source s_all { internal(); unix-stream(quot;/dev/logquot;); file(quot;/proc/kmsgquot; log_prefix(quot;kernel: quot;)); udp(); };
  13. 13. Defining Filters • Windows Filter • Cisco Filter
  14. 14. Windows Filter /etc/syslog-ng/syslog-ng.conf filter f_windows { program(MSWinEventLog); };
  15. 15. Cisco Filter /etc/syslog-ng/syslog-ng.conf filter f_cisco_pix { host(IP.OF.PIX.DEVICE); };
  16. 16. General Filter /etc/syslog-ng/syslog-ng.conf filter f_not_others { not host(IP.OF.PIX.DEVICE) and not program(MSWinEventLog); };
  17. 17. Destinations • FIFO Buffers • One Large File
  18. 18. Windows FIFO /etc/syslog-ng/syslog-ng.conf destination d_windows { pipe(“/var/log/buffers/windows”); };
  19. 19. Cisco FIFO /etc/syslog-ng/syslog-ng.conf destination d_cisco { pipe(“/var/log/buffers/cisco”); };
  20. 20. General FIFO /etc/syslog-ng/syslog-ng.conf destination d_gen_fifo { pipe(“/var/log/buffers/syslog”); };
  21. 21. ...And the Archive /etc/syslog-ng/syslog-ng.conf destination d_all { file(“/var/log/arch/$MONTH$DAY$YEAR”); };
  22. 22. Tying it all Together! • Now we tell syslog to handle the configs. ;)
  23. 23. Windows Log /etc/syslog-ng/syslog-ng.conf log { source(s_all); filter(f_windows); destination(d_windows); };
  24. 24. Cisco Log /etc/syslog-ng/syslog-ng.conf log { source(s_all); filter(f_cisco_pix); destination(d_cisco); };
  25. 25. General FIFO /etc/syslog-ng/syslog-ng.conf log { source(s_all); filter(f_not_others); destination(d_gen_fifo); };
  26. 26. Archive Log /etc/syslog-ng/syslog-ng.conf log { source(s_all); destination(d_all); };
  27. 27. Finishing up... • Making the FIFO buffers • Creating the directory structure
  28. 28. Run me :) $ sudo mkdir /var/log/arch $ sudo mkdir /var/log/buffers $ sudo mkfifo /var/log/buffers/windows $ sudo mkfifo /var/log/buffers/cisco $ sudo mkfifo /var/log/buffers/syslog
  29. 29. Restart Syslog-ng $ sudo /etc/init.d/syslog-ng restart
  30. 30. Is it working? • Check your Logfiles (/var/log/arch/*) • Check your FIFO Buffers • cat /var/log/buffers/windows • cat /var/log/buffers/cisco • cat /var/log/buffers/syslog
  31. 31. Awsome! Wait.... • How are we gonna view this data?
  32. 32. splunk> • Web-based Interface • Indexes arbitrary data • Searchable • Reporting
  33. 33. splunk> • No, I don’t work for them...I just really like their product.
  34. 34. Installing splunk> • Download The latest version (3.0b3 as of writing) • Extract the tarball • Run the application • Make it startup with a system boot
  35. 35. Installing splunk> $ wget 'http://www.splunk.com/index.php/ download_track?file=/3.0b3/linux/ splunk-3.0b3-20872-Linux- i686.tgz&ac=&wget=true&name=wget' $ sudo mkdir /opt;cd /opt $ sudo tar xzvf ~/splunk-3.0b3-20872-Linux- i686.tgz $ sudo /opt/splunk/bin
  36. 36. Configuring splunk>
  37. 37. Configuring splunk>
  38. 38. Configuring splunk>
  39. 39. Configuring splunk>
  40. 40. Configuring splunk>
  41. 41. splunk>
  42. 42. Syslog Agents • Windows Agents • UNIX Agents • Other Devices
  43. 43. Windows Logs? • SNARE Agent • Converts Event Logs to Syslog • Free
  44. 44. UNIX Agents • Use the syslog service! • *.* @Syslog Server
  45. 45. Other Devices • Various systems can be configured • Cisco, Juniper, Lotus Domino, Apache, IIS, etc. are just a few examples.
  46. 46. Recap • What is Syslog • What is FIFO • Installing and Configuring Syslog-NG • Installing and Configuring Splunk • Agents
  47. 47. Questions?

×