Usage Notes of The Bro 2.2 / 2.3

3,941 views

Published on

Notes for using Bro and its Broccoli library.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,941
On SlideShare
0
From Embeds
0
Number of Embeds
82
Actions
Shares
0
Downloads
48
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Usage Notes of The Bro 2.2 / 2.3

  1. 1. Usage Notes of The Bro 2.2 / 2.3 (a network security monitor) William.L wiliwe@gmail.com 2015-02-17
  2. 2. Index Basic Information.................................................................................................................................................. 3 Architecture & System Structure ........................................................................................................................ 4 Install Bro .............................................................................................................................................................. 6 Running Bro Without Installing.................................................................................................................. 8 Use Bro Tools......................................................................................................................................................... 9 Inspect Log Files ......................................................................................................................................... 12 Script Files ................................................................................................................................................... 12 Add Network Application Filter Script..................................................................................................... 14 Read Packet Capture (PCAP) Files........................................................................................................... 15 Communicate With Bro System By Programming.......................................................................................... 17 Default Listen Port Number for Broccoli ................................................................................................. 17 Data Type Mapping between Bro Script and Broccoli Program............................................................ 17 Broccoli Library Documentation............................................................................................................... 18 Broccoli Library Path Setting under 64-bit Environment...................................................................... 20 Reference ............................................................................................................................................................. 22
  3. 3. Basic Information The Bro official site - https://www.bro.org/index.html. Bro is a powerful, passive, open-source network traffic analyzer and analysis framework that is much different from the typical IDS (Intrusion detection system) you may know. It is NOT a classic signature-based IDS (A signature based IDS will monitor packets on the network and compare them against a database of signatures or attributes from known malicious threats. This is similar to the way most antivirus software detects malware.) Well grounded in more than 15 years of research, Bro has successfully bridged the traditional gap between academia and operations since its inception. Bro has originally been developed by Vern Paxson (http://www.icir.org/vern/).
  4. 4. Architecture & System Structure Bro was built based on event-based model. Bro is layered into two major components: event engine and script interpreter. Its event engine (or core) reduces the incoming packet stream into a series of higher-level events. These events reflect network activity in policy-neutral terms, i.e., they describe what has been seen, but not why, or whether it is significant. The event however does not convey any further interpretation, e.g., of whether that URI corresponds to a known malware site; it is done by Bro’s second main component, the script interpreter. The script interpreter which executes a set of event handlers written in Bro’s custom scripting language. These scripts can express a site’s security policy, i.e., what actions to take when the monitor detects different types of activity. More generally they can derive any desired properties and statistics from the input traffic. Bro’s language comes with extensive domain-specific types and support functionality; and, crucially, allows scripts to maintain state over time. Bro scripts can generate real-time alerts and also execute arbitrary external programs on demand, e.g., to trigger an active response to an attack. Bro system contains below components/tools: Component Description Source Folder BinPAC A protocol parser generator. Bro-Src-Root/aux/binpac
  5. 5. bro-aux Small auxiliary tools for Bro. Bro-Src-Root/aux/bro-aux Broccoli The Bro Client Communication Library. Bro-Src-Root/aux/broccoli BroControl An interactive shell for managing Bro installations. Bro-Src-Root/aux/broctl broccoli-python Broccoli Python Bindings. Bro-Src-Root/aux/broccoli/bindings/broccoli-python broccoli-ruby Broccoli Ruby Bindings. Bro-Src-Root/aux/broccoli/bindings/broccoli-ruby BTest A unit testing framework. Bro-Src-Root/aux/btest capstats A command-line tool collecting packet statistics. Bro-Src-Root/aux/broctl/aux/capstats PySubnetTree A Python module for CIDR lookups. Bro-Src-Root/aux/broctl/aux/pysubnettree trace-summary A script generating break-downs of network traffic. Bro-Src-Root/aux/broctl/aux/trace-summary P.S: "Bro-Src-Root" used here is the Bro source folder. Take mine for example, "/home/william/bro-2.3.2"
  6. 6. Install Bro Here using install-from-source way and steps to build and install Bro tools are described in below link. It also lists required and optional dependencies for building/compiling Bro source. https://www.bro.org/sphinx/install/install.html All operations are done under the Linux distribution Ubuntu 14.04 LTS 64-bit. 1) Download a copy of Bro source archive from the official site shown as below and extract the archive. https://www.bro.org/download/index.html Or using GIT to retrieve Bro source: git clone --recursive git://git.bro.org/bro The version used in this document is v2.3.2 (bro-2.3.2.tar.gz) 2) Change directory to Bro source root folder (here using "/home/william/bro-2.3.2" for example), configure building environment and make(compile). There may be auxiliary tools and libraries available in the aux/ sub-directory. Some of them will be automatically built and installed along with Bro. cd /home/william/bro-2.3.2 ./configure --prefix=/home/william/bro [Note: Cause to that the execution of Bro tools needs root privileges mode, so I configure it to install tools into a folder named "bro" in my home directory. If you do not use a folder for installation, it will create folders needed for Bro under your home directory. The default installation path is /usr/local/bro]
  7. 7. make make install If you want to uninstall Bro files(only remove script files), you could run below command(cause to that it uses the Makefile in the sub-directory "build" of Bro source folder. make -C build uninstall Set the environemnt variable PATH to include the path to your installed Bro tools.
  8. 8. Ex: export PATH=$PATH:/home/william/bro/bin Running Bro Without Installing For developers that wish to run Bro directly from the build/ directory (i.e., without performing make install), they will have to first adjust BROPATH environment variable to look for scripts and additional files inside the build directory. Sourcing either build/bro-path-dev.sh or build/bro-path-dev.csh as appropriate for the current shell accomplishes this and also augments your PATH environment variable so you can use the Bro binary directly: ./configure make source build/bro-path-dev.sh bro <options> <script-file>
  9. 9. Use Bro Tools These are the basic configuration(configure files are under the folder Bro-Install-Path/etc ) changes to make for a minimal BroControl installation that will manage a single Bro instance on the localhost: # In Bro-Install-Path/etc/node.cfg, set the network interface for monitoring. The variable for setting interface is “interface” and the network interface name could be found through running command “ifconfig -a”. # In Bro-Install-Path/etc/networks.cfg, comment out the default settings and add the networks that Bro will consider local to the monitored environment.
  10. 10. # In Bro-Install-Path/etc/broctl.cfg, change the MailTo variable for email address to a desired recipient and the LogRotationInterval variable to a desired log archival frequency/period value. 1) Start the BroControl shell, type the command: $ broctrl 2) When you run BroControl shell for the first time, please perform an initial installation of the BroControl configuration: [BroControl] > install 3) Then start up a Bro instance: [BroControl] > start Note: <I> If you encounter a error whose message is similar to below, it means you need root privileges "error: cannot acquire lock: [Errno 13] Permission denied: '/usr/local/bro/spool/lock" <II> If it shows message, "bro terminated immediately after starting", it means there have errors and
  11. 11. you could view the detail through the command "diag." [BroControl] > diag , or you could inspect the error log file “Bro-Install-Path/logs/current/stderr.log.” <II> The user starting BroControl needs permission to capture network traffic. If you are not root, you may need to grant further privileges to the account you’re using. Follow the question and answer of the Bro's FAQ Web page, https://www.bro.org/documentation/faq.html : When bro executable runs normally, you could use ps command to observe it. To stop this Bro instance you would do: Q: How can I capture packets as an unprivileged user? A: Fully implemented since Linux kernel 2.6.24, capabilities are a way of parceling super user privileges into distinct units. Attach capabilities required to capture packets to the bro executable file like this: sudo setcap cap_net_raw,cap_net_admin=eip /path/to/bro where "bro" is a Bro executable tool. Example: sudo setcap cap_net_raw,cap_net_admin=eip /home/william/bro/bin/bro Now any unprivileged user should have the capability to capture packets using Bro provided that they have the traditional file permissions to read/execute the bro binary.
  12. 12. [BroControl] > stop Inspect Log Files By default, logs are written out in human-readable (ASCII) format and data is organized into columns (tab-delimited). Logs that are part of the current rotation interval are accumulated in "Bro-Install-Path/logs/current/" (if Bro is not running, the directory will be empty). By default, BroControl regularly takes all the logs from "Bro-Install-Path/current/" and archives them to a directory named by date, e.g. Bro-Install-Path/logs/2011-10-06. For example: The frequency at which this is done can be configured via the LogRotationInterval option in Bro-Install-Path/etc/broctl.cfg. Some logs are worth explicit mention: conn.log Contains an entry for every connection seen on the wire, with basic properties such as time and duration, originator and responder IP addresses, services and ports, payload size, and much more. This log provides a comprehensive record of the network’s activity. notice.log Identifies specific activity that Bro recognizes as potentially interesting, odd, or bad. In Bro-speak, such activity is called a “notice”. Script Files Bro includes an event-driven scripting language that provides the primary means for an organization to extend
  13. 13. and customize Bro’s functionality. Virtually all of the output generated by Bro is, in fact, generated by Bro scripts. It’s almost easier to consider Bro to be an entity behind-the-scenes processing connections and generating events while Bro’s scripting language is the medium through which we mere mortals can achieve communication. Bro scripts effectively notify Bro that should there be an event of a type we define, then let us have the information about the connection so we can perform some function on it. Bro ships with many pre-written scripts that are highly customizable to support traffic analysis for your specific environment. By default, these will be installed into Bro-Install-Path/share/bro/ and can be identified by the use of a ”.bro“ file name extension. These files should never be edited directly as changes will be lost when upgrading to newer versions of Bro. The exception to this rule is the directory Bro-Install-Path/share/bro/site/ where local site-specific files can be
  14. 14. put without fear of being clobbered later. The other main script directories under Bro-Install-Path/share/bro/ are base and policy. By default, Bro automatically loads all scripts under base (unless the -b command line option is supplied), which deal either with collecting basic/useful state about network activities or providing frameworks/utilities that extend Bro’s functionality without any performance cost. Scripts under the policy directory may be more situational or costly, and so users must explicitly choose if they want to load them. The main entry point for the default analysis configuration of a standalone Bro instance managed by BroControl is the "Bro-Install-Path/share/bro/site/local.bro" script. Adding customized process into this script file. Bro has script packages (e.g. collections of related scripts in a common directory). If the package directory contains a "__load__.bro" script, it supports being loaded in mass as a whole directory for convenience. Packages/scripts in the "base/" directory are all loaded by default, while ones in "policy/" provide functionality and customization options that are more appropriate for users to decide whether they’d like to load it or not. If one wants Bro to be able to load scripts that live outside the default directories in Bro’s installation root, the BROPATH environment variable will need to be extended to include all the directories that need to be searched for scripts. Add Network Application Filter Script Under the folder "Bro-Install-Path/share/bro/policy/misc/app-stats/plugins" 1) Copy a Bro script to a new one. For example: cp facebook.bro amazon.bro 2) Change the filtering condition for Amazon site.
  15. 15. 3) In "__load__.bro", add a line "@load ./amazon" 4) Using Web browser to link to Amazon site, wait for a while and view the log file ”Bro-Install-Path/logs/current/app_stats.log.“ Read Packet Capture (PCAP) Files Capturing packets from an interface and writing them to a file can be done like this: sudo tcpdump -i en0 -s 0 -w mypackets.trace where en0 can be replaced by the correct interface for your system as shown by e.g. ifconfig. (The -s 0 argument tells it to capture whole packets; in cases where it’s not supported use -s 65535 instead). After a while of capturing traffic, kill the tcpdump (with ctrl-c), and tell Bro to perform all the default analysis on the capture which primarily includes : Or, you could use Wireshark/Ethereal(Linux/Windows) or Microsoft Network Monitor(Windows) to capture and saved packets into a PCAP format file. (P.S: Microsoft Network Monitor - http://www.microsoft.com/en-us/download/details.aspx?id=4865) Run below command to read PCAP file and Bro will output log files into the working directory. bro -r mypackets.trace For example:
  16. 16. If you are interested in more detection, you can again load the local script that we include as a suggested configuration: bro -r mypackets.trace local To view the filtering result for application cat app_stats.log
  17. 17. Communicate With Bro System By Programming Q: What is Broccoli? A: BRO Client COmmunications LIbrary. It allows you to write applications that speak the communication protocol of the Bro intrusion detection system for exchanging Bro events with external programs. Broccoli is free software under terms of the BSD license as given in the COPYING file distributed with its source code. From my experiment result of using Bro v2.3 Broccoli and the below link, I changed to use the version of Broccoli of Bro v2.2. http://bro.bro-ids.narkive.com/XaJeX1aM/broccoli-not-processing-events Default Listen Port Number for Broccoli The default port number for listening Broccoli connecting request is 47760 and it could be confirmed through running netstat tool: netstat -ant If you want to change the port number, it could change the port number value in the Python file “options.py” under Bro installation path. After changing the port number, run “broctl” to invoke Bro controller, execute “install” after prompt “[BroControl] >” to re-generate configuration file and then execute “restart” to restart Bro daemon. Using netstat tool to verify if the port number is what you set. Data Type Mapping between Bro Script and Broccoli Program When you want to test an event provided by Bro scripts(.bro file) in program, it needs to convert data type of parameters in event handler of Bro script. Bro official site provides this mapping shown as below:
  18. 18. Broccoli Library Documentation If you want to browse Broccoli library in detail, it provides documentations that could be generated through Doxygen (http://www.stack.nl/~dimitri/doxygen/). In the Broccoli source folder, there has a sub-folder named “doc” containing Doxygen configuration file named “Doxyfile” Change directory to the “doc” sub-folder and run following command to generate HTML-based broccoli documentation: doxygen ./Doxyfile
  19. 19. After the generating process accomplished, it create a folder named “html” under “doc” sub-folder. In “html” folder, the main Web page is index.html, open it in your Web browser to browse Broccoli’s data structures and functions.
  20. 20. Broccoli Library Path Setting under 64-bit Environment In 64-bit Linux, it needs to set Broccoli SO (dynamic) library path manually, otherwise a Broccoli application will fail to run as below. It could use ldd (List Dynamic Dependencies) tool to see what resulted in this fail: the loader could not find the location to the Broccoli SO library, libbroccoli.so. To resolve this problem, it needs to set Broccoli SO library path properly. In CentOS, add soft links to Broccoli SO library files under folder /usr/lib64. In Ubuntu, add soft links to Broccoli SO library files under the folder /lib/x86_64-linux-gnu or /use/lib/x86_64-linux-gnu. After setting correct path to Broccoli, it could use ldd tool to verify again.
  21. 21. Reference * Official Site https://www.bro.org/ * On-line Reference/Documentation https://www.bro.org/sphinx/ * Broccoli library # https://www.bro.org/sphinx/components/broccoli/README.html # https://www.bro.org/sphinx/components/broccoli/broccoli-manual.html * The paper for Bro IDS <I> http://www.icir.org/vern/papers/bro-CN99.html <II> ftp://ftp.ee.lbl.gov/papers/bro-CN99-new.pdf.gz

×