Your SlideShare is downloading. ×
0
Building Centralized
  Logging: Syslog
    Steven “Maniac” McGrath
Syslog?


• logging service
• UNIX based
• Networkable
Wait a Sec...Network?


• UDP port 514
• Typically limited to 1024bytes
One more thing...

• FIFO Buffers
 • First In First Out
 • Rolling View of Logs
 • Type of Named Pipe
FIFO...Tasty *chomp*
     3 Line FIFO Buffer
          Item 5

          Item 4
          Item 3
          Item 2

       ...
Getting Started...


• Ubuntu 6.06 Server
• Base Install
Installing Syslog...
• Update The Repository
Upgrade the OS
• We need to upgrade the OS to current.
Install Syslog-NG
• Syslog-NG will remove klogd, this is normal.
Reconfiguring Syslog-ng
• Configuration depends on network
  environment.
 • Windows Hosts
 • Cisco Devices
 • Linux Hosts
 ...
First off...Global!
                          /etc/syslog-ng/syslog-ng.conf
                            options {
        ...
Next, The Source
       /etc/syslog-ng/syslog-ng.conf

source s_all {
    internal();
    unix-stream(quot;/dev/logquot;);...
Defining Filters


• Windows Filter
• Cisco Filter
Windows Filter
   /etc/syslog-ng/syslog-ng.conf


filter f_windows {
   program(MSWinEventLog);
};
Cisco Filter
   /etc/syslog-ng/syslog-ng.conf


filter f_cisco_pix {
    host(IP.OF.PIX.DEVICE);
};
General Filter
   /etc/syslog-ng/syslog-ng.conf


filter f_not_others {
    not host(IP.OF.PIX.DEVICE)
    and not program...
Destinations


• FIFO Buffers
• One Large File
Windows FIFO
   /etc/syslog-ng/syslog-ng.conf


destination d_windows {
    pipe(“/var/log/buffers/windows”);
};
Cisco FIFO
   /etc/syslog-ng/syslog-ng.conf


destination d_cisco {
    pipe(“/var/log/buffers/cisco”);
};
General FIFO
   /etc/syslog-ng/syslog-ng.conf


destination d_gen_fifo {
    pipe(“/var/log/buffers/syslog”);
};
...And the Archive
   /etc/syslog-ng/syslog-ng.conf


destination d_all {
    file(“/var/log/arch/$MONTH$DAY$YEAR”);
};
Tying it all Together!


• Now we tell syslog to handle the configs. ;)
Windows Log
   /etc/syslog-ng/syslog-ng.conf


log {
    source(s_all);
    filter(f_windows);
    destination(d_windows);
};
Cisco Log
   /etc/syslog-ng/syslog-ng.conf


log {
    source(s_all);
    filter(f_cisco_pix);
    destination(d_cisco);
};
General FIFO
   /etc/syslog-ng/syslog-ng.conf


log {
    source(s_all);
    filter(f_not_others);
    destination(d_gen_f...
Archive Log
   /etc/syslog-ng/syslog-ng.conf


log {
    source(s_all);
    destination(d_all);
};
Finishing up...


• Making the FIFO buffers
• Creating the directory structure
Run me :)

$ sudo mkdir /var/log/arch
$ sudo mkdir /var/log/buffers



$ sudo mkfifo /var/log/buffers/windows
$ sudo mkfif...
Restart Syslog-ng


$ sudo /etc/init.d/syslog-ng restart
Is it working?

• Check your Logfiles (/var/log/arch/*)
• Check your FIFO Buffers
  • cat   /var/log/buffers/windows

  • c...
Awsome! Wait....


• How are we gonna view this data?
splunk>

• Web-based Interface
• Indexes arbitrary data
• Searchable
• Reporting
splunk>


• No, I don’t work for them...I just really like
  their product.
Installing splunk>

• Download The latest version (3.0b3 as of
  writing)
• Extract the tarball
• Run the application
• Ma...
Installing splunk>
$ wget 'http://www.splunk.com/index.php/
download_track?file=/3.0b3/linux/
splunk-3.0b3-20872-Linux-
i6...
Configuring splunk>
Configuring splunk>
Configuring splunk>
Configuring splunk>
Configuring splunk>
splunk>
Syslog Agents


• Windows Agents
• UNIX Agents
• Other Devices
Windows Logs?


• SNARE Agent
 • Converts Event Logs to Syslog
 • Free
UNIX Agents


• Use the syslog service!
• *.*   @Syslog Server
Other Devices


• Various systems can be configured
• Cisco, Juniper, Lotus Domino, Apache, IIS,
  etc. are just a few exam...
Recap

• What is Syslog
• What is FIFO
• Installing and Configuring Syslog-NG
• Installing and Configuring Splunk
• Agents
Questions?
Upcoming SlideShare
Loading in...5
×

Centralized Logging with syslog

7,338

Published on

Published in: Technology, Education
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,338
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
199
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Transcript of "Centralized Logging with syslog"

  1. 1. Building Centralized Logging: Syslog Steven “Maniac” McGrath
  2. 2. Syslog? • logging service • UNIX based • Networkable
  3. 3. Wait a Sec...Network? • UDP port 514 • Typically limited to 1024bytes
  4. 4. One more thing... • FIFO Buffers • First In First Out • Rolling View of Logs • Type of Named Pipe
  5. 5. FIFO...Tasty *chomp* 3 Line FIFO Buffer Item 5 Item 4 Item 3 Item 2 Item 1
  6. 6. Getting Started... • Ubuntu 6.06 Server • Base Install
  7. 7. Installing Syslog... • Update The Repository
  8. 8. Upgrade the OS • We need to upgrade the OS to current.
  9. 9. Install Syslog-NG • Syslog-NG will remove klogd, this is normal.
  10. 10. Reconfiguring Syslog-ng • Configuration depends on network environment. • Windows Hosts • Cisco Devices • Linux Hosts • Other Devices and Gear
  11. 11. First off...Global! /etc/syslog-ng/syslog-ng.conf options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); group(admin); perm(0640); dir_perm(0755); use_dns(no); stats_freq(0); }; • Disable Hostname Chaining • Time to wait before re-establishing a dead connection • Time to wait before an idle file is closed • FIFO Buffer size • Create Directories • Permissions • Disable DNS • Disable Statistic Logging
  12. 12. Next, The Source /etc/syslog-ng/syslog-ng.conf source s_all { internal(); unix-stream(quot;/dev/logquot;); file(quot;/proc/kmsgquot; log_prefix(quot;kernel: quot;)); udp(); };
  13. 13. Defining Filters • Windows Filter • Cisco Filter
  14. 14. Windows Filter /etc/syslog-ng/syslog-ng.conf filter f_windows { program(MSWinEventLog); };
  15. 15. Cisco Filter /etc/syslog-ng/syslog-ng.conf filter f_cisco_pix { host(IP.OF.PIX.DEVICE); };
  16. 16. General Filter /etc/syslog-ng/syslog-ng.conf filter f_not_others { not host(IP.OF.PIX.DEVICE) and not program(MSWinEventLog); };
  17. 17. Destinations • FIFO Buffers • One Large File
  18. 18. Windows FIFO /etc/syslog-ng/syslog-ng.conf destination d_windows { pipe(“/var/log/buffers/windows”); };
  19. 19. Cisco FIFO /etc/syslog-ng/syslog-ng.conf destination d_cisco { pipe(“/var/log/buffers/cisco”); };
  20. 20. General FIFO /etc/syslog-ng/syslog-ng.conf destination d_gen_fifo { pipe(“/var/log/buffers/syslog”); };
  21. 21. ...And the Archive /etc/syslog-ng/syslog-ng.conf destination d_all { file(“/var/log/arch/$MONTH$DAY$YEAR”); };
  22. 22. Tying it all Together! • Now we tell syslog to handle the configs. ;)
  23. 23. Windows Log /etc/syslog-ng/syslog-ng.conf log { source(s_all); filter(f_windows); destination(d_windows); };
  24. 24. Cisco Log /etc/syslog-ng/syslog-ng.conf log { source(s_all); filter(f_cisco_pix); destination(d_cisco); };
  25. 25. General FIFO /etc/syslog-ng/syslog-ng.conf log { source(s_all); filter(f_not_others); destination(d_gen_fifo); };
  26. 26. Archive Log /etc/syslog-ng/syslog-ng.conf log { source(s_all); destination(d_all); };
  27. 27. Finishing up... • Making the FIFO buffers • Creating the directory structure
  28. 28. Run me :) $ sudo mkdir /var/log/arch $ sudo mkdir /var/log/buffers $ sudo mkfifo /var/log/buffers/windows $ sudo mkfifo /var/log/buffers/cisco $ sudo mkfifo /var/log/buffers/syslog
  29. 29. Restart Syslog-ng $ sudo /etc/init.d/syslog-ng restart
  30. 30. Is it working? • Check your Logfiles (/var/log/arch/*) • Check your FIFO Buffers • cat /var/log/buffers/windows • cat /var/log/buffers/cisco • cat /var/log/buffers/syslog
  31. 31. Awsome! Wait.... • How are we gonna view this data?
  32. 32. splunk> • Web-based Interface • Indexes arbitrary data • Searchable • Reporting
  33. 33. splunk> • No, I don’t work for them...I just really like their product.
  34. 34. Installing splunk> • Download The latest version (3.0b3 as of writing) • Extract the tarball • Run the application • Make it startup with a system boot
  35. 35. Installing splunk> $ wget 'http://www.splunk.com/index.php/ download_track?file=/3.0b3/linux/ splunk-3.0b3-20872-Linux- i686.tgz&ac=&wget=true&name=wget' $ sudo mkdir /opt;cd /opt $ sudo tar xzvf ~/splunk-3.0b3-20872-Linux- i686.tgz $ sudo /opt/splunk/bin
  36. 36. Configuring splunk>
  37. 37. Configuring splunk>
  38. 38. Configuring splunk>
  39. 39. Configuring splunk>
  40. 40. Configuring splunk>
  41. 41. splunk>
  42. 42. Syslog Agents • Windows Agents • UNIX Agents • Other Devices
  43. 43. Windows Logs? • SNARE Agent • Converts Event Logs to Syslog • Free
  44. 44. UNIX Agents • Use the syslog service! • *.* @Syslog Server
  45. 45. Other Devices • Various systems can be configured • Cisco, Juniper, Lotus Domino, Apache, IIS, etc. are just a few examples.
  46. 46. Recap • What is Syslog • What is FIFO • Installing and Configuring Syslog-NG • Installing and Configuring Splunk • Agents
  47. 47. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×