www.rti.com                  System Monitoring and              Network Intrusion Detection                     using DDS ...
Background    This presentation describes the results and    architecture of a DoD-funded research effort to    develop a ...
Project Partners     RTI (Prime)   DDS     Coral8   CEP     SL       Visualization     Promia   IDS Hardware     ObjectSec...
Outline      The problem of [Distributed] System Monitoring      – System Monitoring      – Intrusion Detection      Avail...
System Monitoring & Network Intrusion     Middleware Technologies, such as DDS help build large     distributed systems 10...
100’s of Available Tools                                                    See: http://sectools.org/    Tool Categories: ...
The tool Integration problem      Many open source and COTS tools      – Complementary functionality      – Overlapping fu...
Each tool an isolated island                Sensor1    Detection   Visualization1                           Engine1       ...
A better approach!                                                                    Visualization          Snort        ...
A better approach with Standards & COTS     COTS System Sensors COTS IDS Sensors       Custom Sensors          (Ganglia)  ...
Infrastructure Technology Selection       DDS was selected because it provides a       Standard API and Network Protocol a...
Focus:     Network Intrusion & System Monitoring       Network Intrusion Detection System (IDS)        – Sensor: Monitor “...
IDS: Snort Overview      Open source software (Linux, Windows)       – Commercial support available      Can operate as:  ...
Snort deployment     Internet                          Detect intrusion         Detect all       events that get     intru...
Snort Implementation     Outputs alerts     To files, database,     Socket, …     Applies rules     Detects “intrusion”   ...
Snort rules     <action> <proto> <src> <direction> <dst> (<options>)     action   protocol     source           destinatio...
Snort Actions        alert – generate alert and log packet        log - log packet        pass – ignore packet        acti...
Normalized IDS Alert information                                      struct SensorDetails {                              ...
System Monitoring: Ganglia Overview      Open source software (Linux, Windows)       – Originally developed at UC berkeley...
Ganglia Deployment (data collection)         Host-based (gmond daemon on each host)          – monitor changes in host sta...
Ganglia Implementation & Deployment                                                          Apache                       ...
Normalized System Monitoring Information                                       struct ClusterInfo {                       ...
CEP Overview       Capabilities:       • Filtering       • Pattern Detection       • Aggregation, transformation       • T...
CEP – Example 1     activity: ftp request                             DDS Topic         tool: Snort                       ...
CEP – Example 2      activity: ssh                       DDS Topic       tool: Snort                                      ...
Potential      Information normalization would allow rules to be written      independent of sensors      Use of CEP would...
Status: RTI Prototype Demo CONOPS v0.1                   Shapes Traffic                                                   ...
Future objectives       Integrate additional sensors of each class &       refine the Normalized data format       Gather ...
Questions       Gerardo Pardo-Castellote, Ph.D.       gerardo.pardo@rti.comwww.rti.com                              29
Upcoming SlideShare
Loading in …5
×

System monitoring and network intrusion using DDS and CEP

1,723 views

Published on

There are many approaches to System Monitoring and Network Intrusions. In many cases these approaches are complementary, the strengths of one filling for the weakness in others. However as each system operates in isolation it becomes difficult to leverage the combined capabilities.

This presentation describes an approach that is based on the OMG Data Distribution Service standard to normalize and combine the information from multiple system monitoring probes and intrusion detection sensors, and further combine it with the power of off-the-shelf Complex Event Processing Engines (CEP) to develop a holistic, all-incluse approach to system monitoring and intrusion detection.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,723
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
46
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

System monitoring and network intrusion using DDS and CEP

  1. 1. www.rti.com System Monitoring and Network Intrusion Detection using DDS and CEP Gerardo Pardo-Castellote, Ph.D. (RTI) Joe Schlesselman (RTI) OMG RTESS Workshop Washington DC, July 14-16, 2008 1
  2. 2. Background This presentation describes the results and architecture of a DoD-funded research effort to develop a common normalized information picture combining data from System Monitoring Tools and Intrusion Detection Systems. The purpose of this work is to increase the speed and effectiveness of the technologies used to detect & counter network attacks.2
  3. 3. Project Partners RTI (Prime) DDS Coral8 CEP SL Visualization Promia IDS Hardware ObjectSecurity Security Manager3
  4. 4. Outline The problem of [Distributed] System Monitoring – System Monitoring – Intrusion Detection Available Technologies – Ganglia, Nagios – Snort, Nessus, Saint Practical solution using DDS and CEP4
  5. 5. System Monitoring & Network Intrusion Middleware Technologies, such as DDS help build large distributed systems 100s or 1000s or computers with an order of magnitude more processes – Functionally the system may be correct – … but it may still fail operationally… These large systems require monitoring. We focus on two aspects: – Network Intrusion Detection – System Monitoring – Information Sharing/Normalization – Information Processing The integration of these technologies illustrates the essence of the open integration problem we are trying to address.5
  6. 6. 100’s of Available Tools See: http://sectools.org/ Tool Categories: Network Intrusion Detection Systems – Snort – OSSEC NDIS Vulnerability Scanners – Nessus, Nmap – Saint, GFI LanGuard Sniffers – Wireshark, Kismet, – Netcat, Metasplot, hping, OS fingerprinting, Service identification – Nmap, P0f System Monitoring – Ganglia Knowledge Repositories – IDS Rules – Vulnerability Databases (CVE) Analysis, Detection, Decision Making – General purpose (e.g. CEP) – Specialized (e.g. Snort rules) Monitoring, Visualization/HMI Tools – HP OpenView – SL RTView, Promia Asset Viewer6
  7. 7. The tool Integration problem Many open source and COTS tools – Complementary functionality – Overlapping functionality Each tool must combine: – Sensors, Rules, Alerts/Notifications, Visualization… “Each Tool and Isolated Island” – Rules/knowledge bound to the tool/sensor – Must learn specifics of each sensor – Can’t easily multiple sensors in a rule – Limited to what each sensor offers The Result: Cost, Complexity, Limited Power: – Cannot aggregate, correlate, extend7
  8. 8. Each tool an isolated island Sensor1 Detection Visualization1 Engine1 Rules1 Sensor2 Detection Visualization2 Engine2 Rules2 Sensor3 Detection Visualization3 Engine3 Rules38
  9. 9. A better approach! Visualization Snort (3D Asset Viewer) nmap Visualization ossec nessus Archive normalized Ganglia normalizedsensor data Network alarm/event inputs SAINT Management outputs Shared Information Bus / Normalized Information Picture Custom IDS Detection General Analysis Custom Classified Engine Engine Code9 new sensors COTS UTM custom CEP new processors
  10. 10. A better approach with Standards & COTS COTS System Sensors COTS IDS Sensors Custom Sensors (Ganglia) (Snort) Host Sensor Host Sensor COTS Vulnerability Sensors (Nmap, Nessus, Saint) Send updated Vulnerability Signature signature COTS Middleware Database ` (DDS Global Data Space) Custom/Classified IDS Sensors COTS COTS Event Recording/ Processing Auditing10 COTS HMI/ Dashboards Email
  11. 11. Infrastructure Technology Selection DDS was selected because it provides a Standard API and Network Protocol able to handle large volumes of real-time information and prioritize it by setting QoS policies. CEP was selected because it provides a familiar (SQL-like), powerful and extensible language able to process large amounts of streaming data, aggregate the information, correlate it, and uncover interesting events and threats.11
  12. 12. Focus: Network Intrusion & System Monitoring Network Intrusion Detection System (IDS) – Sensor: Monitor “raw” network packets – Process Dissect packets, Identify conversations, Look inside Look for: known attack patterns, scans, unusual activity – Output: Generate Alarms, Filter/drop packets – Example: Snort, OSSEC System Monitor – Sensor: Host sensors, look at CPU, memory, file system, network, … – Process: Detect unusual loads or changes – Output: Status, Alarms – Example: Ganglia System and Service Identification – Sensors: Host sensors, Network packet capture – Output: List of hosts, OS fingerprints, List of open services – Examples: nmap, P0f12
  13. 13. IDS: Snort Overview Open source software (Linux, Windows) – Commercial support available Can operate as: – Sniffer / packet logger, – Inline Mode: Intrusion Prevention System (IPS) Interfaces with IP tables to drop/reject packets/connections and log the alert Can also modify bytes in the packets – Network intrusion detection system (NDIS) Uses lib PCAP to monitor traffic & generate alerts Can also operate from a saved PCAP file Can match src/dst and do deep packet inspection Can produce references to know attack databases (e.g. NIST’s CVE) Simple rule language for NDIS – Customizable/extensible by end user – Active community-based rules database13
  14. 14. Snort deployment Internet Detect intrusion Detect all events that get intrusion events thought the firewall14
  15. 15. Snort Implementation Outputs alerts To files, database, Socket, … Applies rules Detects “intrusion” events Analyzes, modifies and takes actions like: Alert, Log, Drop, Reject.15
  16. 16. Snort rules <action> <proto> <src> <direction> <dst> (<options>) action protocol source destination alert tcp any any -> 10.10.100.0/24 111 (content:“|00 01 86 a5|”; msg:”mountd access”;) keyword keyword The rule matches if all elements match: protocol, src, destination, … content, etc. – <direction> can be “->” or “<>” – “content:” used for deep-packet inspection. There is a mini language: Can specify offsets and depth, Can relate to other matches in same packet Can use PERL regular expressions – Can also do matches in the packet (IP/TCP,UDP) headers If a rule matches the action is taken: – Alert, log, pass,16 Rules are applied in sequence
  17. 17. Snort Actions alert – generate alert and log packet log - log packet pass – ignore packet activate – alert and turn on dynamic rule dynamic – inactive rule until activated then log Inline-mode only: drop – drop packet and log sdrop – drop packet without logging reject – drop packet, log, and send reset/ICMP port unreachable to sender Custom actions can also be used to bind to specific output plugins: Ruletype dds_alert { type alert output dds_write: domain=36 topic=“SnortAlert” }17
  18. 18. Normalized IDS Alert information struct SensorDetails { string name; string version; long agentId; struct NormalizedAlert { long type; SensorDetails sensor; NodeInfo node; NodeInfo source; }; NodeInfo target; struct NodeInfo { ProtocolInfo protocolInfo; NetworkAddr networkAddr; Timestamp alarmTimestamp; unsigned short port; string alertMsg; PhysicalAddress phylAddr; short priority; string hostName; sequence<ExtraValue> extra; }; }; struct ProtocolInfo { string type; UdpInfo udpInfo; TcpInfo tcpInfo; IpInfo ipInfo; IcmpInfo icmpInfo; EthernetInfo ethInfo;18 };
  19. 19. System Monitoring: Ganglia Overview Open source software (Linux, Windows) – Originally developed at UC berkeley Distributed Monitoring system for Clusters/Grids – Has scaled to clusters with 2000 nodes Many built-in metrics – CPU, Network, IO, Memory, Disk – Can be extended with user-defined metrics19
  20. 20. Ganglia Deployment (data collection) Host-based (gmond daemon on each host) – monitor changes in host state – announce relevant changes – listen to the state of other ganglia nodes – answer requests for an XML description of the cluster state. Outputs: – Metrics to multicast UDP Sent at configured period … or when value changes beyond configurable threshold – Responses to requests received via TCP multicast GMOND GMOND GMOND GMOND GMOND GMOND GMETRIC APP20 (standard metrics) (custom metrics)
  21. 21. Ganglia Implementation & Deployment Apache HMI Web Server RR Database GMETAD TCP Poll Cluster 1 RR GMOND GMETAD Database mu ltic P Poll TC sta GMOND Cluster 2 GMOND Cluster 3 GMOND GMOND GMETRIC APP21
  22. 22. Normalized System Monitoring Information struct ClusterInfo { string name; Timestamp localtime; string owner; string latlong; string url; }; struct NormalizedSysMon { SensorDetails sensorDetails; struct StandardMetrics{ ClusterInfo clusterInfo; DoubleMetric disk_total; StandardMetrics stdMetrics; LongMetric cpu_speed; }; LongMetric swap_total; StringMetric os_name; FloatMetric cpu_user; FloatMetric cpu_system; FloatMetric bytes_out; FloatMetric pckts_in; … };22
  23. 23. CEP Overview Capabilities: • Filtering • Pattern Detection • Aggregation, transformation • Time-Correlation … Across multiple streams Real-Time Composite Data Feeds Events ... Alerts CEP Engine Actions Event23 Storage
  24. 24. CEP – Example 1 activity: ftp request DDS Topic tool: Snort RTView DDS Topic CEP CEP RTI activity: disk free Recorder Rule: DDS Topic If free disk < 10% and tool: Ganglia ftp request Then send DDS topic  alert24
  25. 25. CEP – Example 2 activity: ssh DDS Topic tool: Snort RTView DDS Topic CEP CEP RTI event: CPU load Recorder Rule: DDS Topic If CPU > 50% and tool: Ganglia ssh request Then send DDS topic  alert25
  26. 26. Potential Information normalization would allow rules to be written independent of sensors Use of CEP would – allow rules to apply across sensors – enable much more sophisticated rules that mix historical data, time-windows, causality, etc. Use of DDS would – enable prioritization and other QoS – provide unified API to access all data with highest performance – allow segmentation and parallelization of processing – enable easy visualization using general purpose tools – Provide many out of the box services: recording, durability… across all sensors with no additional work26
  27. 27. Status: RTI Prototype Demo CONOPS v0.1 Shapes Traffic Promia Raven UTM Shapes Traffic Generic Traffic SL RTView Windows Generic Traffic CEP via Windows Coral8 Engine Shapes Traffic RTI Developer Shapes Traffic Generic Traffic Processors Platform Tools Solaris (Analyzer, Wireshark) Generic Traffic Solaris Shapes Traffic Generic Traffic RTI DDS Shapes Traffic Generic Traffic Linux RTC Database Producers Linux (SQL, Archive) Ganglia Nmap Consumers Snort Nessus Rogue DDS SAINT Linux OpenPMF Naughtiness Sensors Shapes traffic = UDP/IPv4 DDS traffic created by RTI DDS Shapes demo application Generic Traffic = legitimate and illegitimate random TCP/IPv4 network traffic Legitimate = valid, well-formed packets of various protocols created using network traffic stimulator Illegitimate = invalid, non-well-formed but not malicious packets of various protocols27 Rogue = packets that violate one or more security policies, whether or not malicious
  28. 28. Future objectives Integrate additional sensors of each class & refine the Normalized data format Gather IDS data in real scenarios Develop more sophisticated CEP rules based on IDS data Develop CONOPS v 1.028
  29. 29. Questions Gerardo Pardo-Castellote, Ph.D. gerardo.pardo@rti.comwww.rti.com 29

×