#212148                                                                                                                   ...
IBM IPS GX7800                                                                                                        #212...
IBM IPS GX7800                                                                                                            ...
IBM IPS GX7800                                                                                                            ...
IBM IPS GX7800                                                                                                            ...
IBM IPS GX7800                                                                                                          #2...
Upcoming SlideShare
Loading in …5

Tolly Group Report: IBM Security Network IPS GX7800 Appliance


Published on

This report tested the IBM Security Network IPS GX7800 appliance, certifying that it:

1) Delivers superior protection from evolving threats with high levels of performance.
2) Stops 99% of tested, publicly available attacks.
3) Is nearly twice as effective as Snort at stopping "mutated" attacks.
4) Protects streams of 100% HTTP traffic at speeds of 20 Gbps and mixed traffic loads of 35 Gbps+.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Tolly Group Report: IBM Security Network IPS GX7800 Appliance

  1. 1. #212148 November 2012 Commissioned by IBM Security Systems Division IBM Security Network Intrusion Prevention System GX7800 Comparative Efficacy and Performance EvaluationExecutive Summary The Bottom LineEnterprise-class networks today are facing more advanced threats from amultitude of sources than ever before. Effective threat protection solutions must The IBM Network Security IPS GX7800:defend against real-world threats that are evolving quickly, and at the same timedeliver high levels of performance and availability. IBM commissioned Tolly toevaluate their protocol-based Network Intrusion Prevention System (IPS) GX7800 1 Delivers superior protection from evolving threats with high levels of performanceand compare its efficacy to that of a Snort-based device, a signature-basedplatform. 2 Stopped 99% of tested, publicly-available attacksTolly engineers conducted many different performance tests with the GX7800and achieved a maximum of 35.7 Gbps throughput under mixed trafficloads.  This demonstrates a great tolerance for network surges, growth and 3 Was nearly twice as effective as Snort at stopping mutated attackscapacity over IBMs published performance characteristics. Tolly also evaluatedthe IBM IPS GX7800’s efficacy and functionality.Tests showed the IBM IPS GX7800 to be more effective blocking publicly-available exploits than Snort and dramatically more effective when blocking 4 Protected streams of 100% HTTP traffic at speeds of 20 Gbps and mixed traffic loads atmutated exploits - blocking 100% compared to 52% for Snort. See Figure 1. over 35 Gbps Inline IPS System Efficacy Against Publicly-Available (PA) and Mutated Exploits IBM IPS GX7800 vs. Snort IPS Publicly-Available Exploits Blocked Mutated Exploits Blocked (Out of 74) (Out of 31) 99% 100 % 91 % Exploits Blocked (%) Exploits Blocked (%) 52% IBM IPS GX7800 Snort IBM IPS GX7800 Snort IBM IPS GX7800 Snort Source: Tolly, October 2012 Figure 1© 2012 Tolly Enterprises, LLC Tolly.com Page 1 of 6
  2. 2. IBM IPS GX7800 #212148As enterprise IT has evolved, network Mutated Threats Blockedsecurity should keep pace. Today’s threats are As with the AV industry, the Internet is host IBM Securitymore refined, diverse, and potentiallyharmful than ever—and as a result they to an ever-expanding number of threats. Systemsrequire new and intuitive solutions to offset You can think of signature-based solutions as Divisiontheir negative impact. a face recognition system and the mutationTraditional signature-based IPS solutions as a mask that “mutates” the face and can IPS GX7800don’t protect against the evolving threats confuse the face recognition system.that are ever-present in today’s enterprise Signature-based solutions have difficulty Efficacy andenvironment. Signature-based IPS solutions keeping pace when threats are mutating by Performance Testedcan protect against an exploit once it is the thousands. In order to replicate these Evaluation Octoberknown, but offers less protection against mutations, engineers deliberately altered thethreats that have mutated . payloads of the tested exploits. This was 2012Using its protocol analysis module (PAM), the accomplished in most cases by changing theIBM GX7800 is able to decode the name of a single variable within the exploit With 44K objects, the IBM GX7800 deliveredapplication traffic and identify malicious code. over 19 Gbps in “drop” mode and more thancode in any form, helping to maintain a more The IBM GX7800 stopped 100% of mutated 24 Gbps in “forward” mode.secure network than signature-based IPS threats, while the signature-based Snort The IBM GX7800 delivered identical results inalone. Furthermore, the engine is extensible solution stopped half (16 out of 31) of the both modes for Core IPS and Enterprise IPSand can cover more than just vulnerabilities mutated exploits. See Figure 1. traffic profiles, demonstrating 35.7 Gbps of(e.g SQL injection and shell code). The IBM throughput for all four scenarios (Core IPSGX7800 is only part of the solution that IBM Performance Test Results drop/forward and Enterprise IPS drop/provides. Behind the scenes, IBM’s X-Force forward). See Figure 2.Research and Development Team In today’s enterprise environment, security is a must. However, performance is just asproactively seeks out new threats, important for large deployments. Features/Functionalityincorporating this insight back to theappliance via software updates. Organizations need to remain online and Though some features can be viewed as secure at multiple 10GbE speeds. “nice to have”, a certain usability of anTest Results Engineers verified the performance of the IBM GX7800 using Ixia’s BreakingPoint effective system should not be overlooked. The IBM GX7800 provides a variety of features/functions that make its deploymentEfficacy Test Results FireStorm in both “drop” and “forward” modes across a range of object sizes that and management intuitive and easy-to-use.Publicly-Available Threats Blocked included streams of pure HTTP traffic as well From the dashboard, administrators are as streams containing mixes of enterprise greeted with an “at-a-glance” look into theIBM X-Force gathered exploits from the X- and core traffic types. overall network security, including recentForce Database where they publish all events, and general statistics for many of thedisclosed vulnerabilities and exploits from Tolly test results show that the IBM GX7800 modules and threats.many sources. can maintain high levels of performance in both “drop” and “forward” modes. Testing By default, the IBM GX7800 is equipped withTolly engineers tested the IBM GX7800 and with “drop” mode enabled disallows any powerful policies in the form of X-Forcethe open-source Snort1 device against a traffic beyond what the device could scan at Virtual Patch, a collection of protectioncorpus of 74 such threats. The IBM GX7800 a given time, whereas with “forward” mode algorithms which are specifically designed tostopped 99% (73 out of 74) of the exploits, enabled, excess traffic is forwarded to the not generate false positives and act as awhile the open-source Snort device blocked network without having been scanned, but stand-in for many application patches.only 67 out of 74. See Figure 1. is important for business continuity.1 Snort® is an open source network intrusion prevention and detection system (IDS/IPS) developed by Sourcefire. © 2012 Tolly Enterprises, LLC Tolly.com Page 2 of 6
  3. 3. IBM IPS GX7800 #212148 IBM IPS GX7800 Aggregate System Throughput 8X10GbE Ports In Drop Mode and Forward Mode As reported by Ixia BreakingPoint FireStormAggregate Data Rate (Gbps) 35.7 35.7 35.7 35.7 20.8 24.1 19.5 12.0 10.3 7.8 10K 21K 44K Core IPS Enterprise IPS HTTP Traffic (100%) Mixed Traffic (HTTP, FTP, SMTP, DNS, etc) Source: Tolly, October 2012 Drop Mode Testing Forward Mode Testing Figure 2In addition to providing high-performance Testing was conducted within a VMware ESXi The Snort instance was configured with thesecurity, the GX7800 is also host to an array 5.0.0 environment. Multiple clients were latest VRT (Vulnerability Research Team)of other functions including a simple deployed and configured to be vulnerable to updates as of October 16, 2012, and wasNetwork Data Loss Prevention (NDLP) the exploits tested. Snort was deployed on a running Snort Engine version, set tomodule and web application protection. CentOS client with 1vCPU and 2GB RAM. inline mode. To ensure the highest securityThese inclusions transform the GX7800 into Multiple VM Networks were created within settings were used, engineers configured allan all-purpose network security appliance. the host to segment the “Attacker” network Snort Rules and SO_Rules to “Block” byThe GX7800 can also be deployed in high from the “Vulnerable” network. default. Both the plaintext rules andavailability scenarios within an organization obfuscated, shared object rules were usedto further increase the amount of statefultraffic which can be inspected. Likewise,multiple appliances can be deployed in IBM Security Network Intrusion Prevention Solutionsgeographically disparate sites within an Product Specificationsorganization. In this topology, appliances Performance Characteristicsshare information with each other and can • 200 Mbps to 20 Gbp + aggregated throughput (depending on model)be centrally managed from a single console. • 1.3M to 21M simultaneous connections (depending on model) • Less than 150 microseconds of latency (less than 75 microseconds for GX7 models)Test Setup & Core Capabilities Availability • Virtual Patch technology • Active/active high availabilityMethodology • Web application protection • Redundant hard drives and power supplies • Protection from client-side attacks Research/UpdatesTest Bed Setup • Data and content security • Application awareness • Updates powered by IBM X-Force research teamEngineers deployed an environment • X-Press Updates - automated updated delivery Protection Modes Management Optionsconsisting of one IBM Network Security • In-line protection • Local web-based managementIntrusion Prevention System GX7800, which • In-line simulation • Centralized management via IBM SiteProtectorwas equipped with 8x10GbE ports, running • Passive monitoringfirmware v4.5 with security content XPU For more information, call 1-877-426-3774 or visit:version 32.090. http://www-01.ibm.com/software/tivoli/products/security-network-intrusion-prevention/ Source: IBM © 2012 Tolly Enterprises, LLC Tolly.com Page 3 of 6
  4. 4. IBM IPS GX7800 #212148for the test. Snort claims coverage for all headers, with a time delay of 2 seconds Tests were run for 5 minutes in both Dropcommon vulnerabilities and exposures (CVE) between trace replays. and Forward mode configured on the IBMused in testing. GX7800. Due to the nature of the All 74 base exploits and 31 mutations were BreakingPoint traffic, certain DOS attacks run through the environment andTest Methodology repeatable results were reported by Traffic IQ. would occasionally trigger when tests were running. For the purpose of the performanceFor the efficacy testing, Metasploit For the Performance tests, Tolly utilized an testing, these rules were disabled to allowFramework 4.5.0-dev-15713 was used to Ixia BreakingPoint FireStorm system version the traffic to flow without error.create payloads and deliver the exploits to 3.0 b105019. Engineers tested three HTTPvulnerable hosts. Publicly-available exploits Engineers also injected a 6-attack StrikePack object sizes (10K, 21K, 44K) with 250 clients/were used for all CVEs. ‘Mutants’ were to verify that attacks were being detected servers per AppSim profile, with two on eachexploits with various changes to the code while under heavy load. In no scenario were port to get bidirectional transactions.(such as changing variable/function names) attacks allowed through the GX7800. Additionally, engineers used the Core andwhich produced the same outcome as the Enterprise IPS traffic mixes to stress theoriginal exploit. This is a common approach GX7800. These mixes contained HTTP, SMTP,used to attack systems. See Figure 4. SIP, FTP, DNS and other stateful traffic.Initially, exploits were run without anysecurity solution inline and packet captures Test Equipment Summarywere created for both sides of the Vendor Product Webconversation.Idappcom Traffic IQ Professional v2.0.299was used to replay both sides of the Ixia BreakingPoint Ixiaconversation through interfaces on the FireStorm V3.0 http://www.ixiacom.comvulnerable and attacker networks while eachIPS device was connected in inline mode.Traffic IQ was configured to rewrite HTTP Intrusion Prevention System Test Environment Source: Tolly, October 2012 Note: For Snort testing, the IBM IPS was replaced by the device running Snort. Figure 3 © 2012 Tolly Enterprises, LLC Tolly.com Page 4 of 6
  5. 5. IBM IPS GX7800 #212148 Test Data: Mutation Examples and Common Vulnerabilities Mutation Example 1: Renaming Variables Many of the tested exploits contain variable names. These variable names are irrelevant to successful exploitation, and therefore cannot be depended upon for detection of exploit attempts. In order to test mutated versions of the exploits, we simply altered the variables names, as shown in the examples below: Original Variable Names Mutated Variable Names Shellcode somecode Block brick heapLib badLib While these changes had no impact on the effectiveness of the exploits, they did allow the exploits to go undetected by the signature- based Snort solution. Mutation Example 2: Renaming Class References Many of the tested exploits contain references to classes contained within Java archives. Because class filenames within an archive are variable and arbitrary, they should not be depended upon for detection of malicious activity. In order to test mutated versions of the exploits, we simply altered the referenced class names, as shown in the example below: Original Class Reference Mutated Class Reference <html><head></head> <html><head></head> <body><applet archive="jmBXTMuv.jar" <body><applet archive="eXRZLr.jar" code="msf.x.badguy.class" code="msf.x.Exploit.class" width="1" height="1"><param width="1" height="1"><param name="data" value=""/><param name="data" value=""/><param name="jar"> name="jar"> While these changes had no impact on the effectiveness of the exploits, they did allow the exploits to go undetected by the signature- based Snort solution. Mutation Example 3: Adding Comments In order to test mutated versions of some of the exploits, we simply added comments to the exploit code as shown in the example below: Original Code Mutated Code var t = unescape; var t = unescape <!— Comment -->; While these changes had no impact on the effectiveness of the exploits, they did allow the exploits to go undetected by the signature- based Snort solution. Tested Server Vulnerabilities CVE-2012-0002 CVE-2011-4191 CVE-2011-3192 CVE-2011-1248 CVE-2011-1206 CVE-2011-0807 CVE-2011-0654 CVE-2011-0267 CVE-2011-0266 CVE-2010-3972 CVE-2010-2729 CVE-2010-1555 CVE-2010-0478 CVE-2009-3103 CVE-2009-3023 CVE-2009-1429 CVE-2008-4250 CVE-2008-1697 Tested Client Vulnerabilities CVE-2012-1889 CVE-2012-1875 CVE-2012-0779 CVE-2012-0507 CVE-2012-0500 CVE-2012-0158 CVE-2012-0013 CVE-2011-3544 CVE-2011-3400 CVE-2011-2462 CVE-2011-1260 CVE-2011-0611 CVE-2011-0609 CVE-2011-0105 CVE-2011-0073 CVE-2011-0065 CVE-2011-0041 CVE-2011-0027 CVE-2010-4452 CVE-2010-3971 CVE-2010-3970 CVE-2010-3962 CVE-2010-3654 CVE-2010-3653 CVE-2010-3552 CVE-2010-3333 CVE-2010-3148 CVE-2010-2883 CVE-2010-2568 CVE-2010-1885 CVE-2010-1423 CVE-2010-1297 CVE-2010-1240 X2 CVE-2010-0842 CVE-2010-0840 CVE-2010-0806 CVE-2010-0805 CVE-2010-0249 CVE-2010-0248 CVE-2010-0188 CVE-2010-0094 CVE-2010-0033 CVE-2010-0027 CVE-2009-4324 CVE-2009-3459 CVE-2009-2477 CVE-2009-1534 CVE-2009-1136 CVE-2009-0927 CVE-2009-0658 CVE-2009-0075 CVE-2008-4844 CVE-2008-4037 CVE-2008-2992 CVE-2008-0015Note: To view details of a given CVE, use the following format with the CVE name:: http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-1206Source: Tolly, October 2012 Figure 4© 2012 Tolly Enterprises, LLC Tolly.com Page 5 of 6
  6. 6. IBM IPS GX7800 #212148 About Tolly Interaction with Competitors The Tolly Group companies have been delivering world-class IT services for In accordance with Tolly’s Fair Testing Charter, Tolly personnel invited more than 20 years. Tolly is a leading representatives from Sourcefire, Inc, developer of Snort, to participate in global provider of third-party validation services for vendors of IT the testing. Sourcefire reviewed the test plan and declined to participate. products, components and services. For more information on the You can reach the company by E-mail Tolly Fair Testing Charter, visit: at sales@tolly.com, or by telephone at http://www.tolly.com/FTC.aspx +1 561.391.5610. Visit Tolly on the Internet at: http://www.tolly.com Terms of Usage This document is provided, free-of-charge, to help you understand whether a given product, technology or service merits additional investigation for your particular needs. Any decision to purchase a product must be based on your own assessment of suitability based on your needs. The document should never be used as a substitute for advice from a qualified IT or business professional. This evaluation was focused on illustrating specific features and/or performance of the product(s) and was conducted under controlled, laboratory conditions. Certain tests may have been tailored to reflect performance under ideal conditions; performance may vary under real-world conditions. Users should run tests based on their own real-world scenarios to validate performance for their own networks. Reasonable efforts were made to ensure the accuracy of the data contained herein but errors and/or oversights can occur. The test/ audit documented herein may also rely on various test tools the accuracy of which is beyond our control. Furthermore, the document relies on certain representations by the sponsor that are beyond our control to verify. Among these is that the software/ hardware tested is production or production track and is, or will be, available in equivalent or better form to commercial customers. Accordingly, this document is provided "as is", and Tolly Enterprises, LLC (Tolly) gives no warranty, representation or undertaking, whether express or implied, and accepts no legal responsibility, whether direct or indirect, for the accuracy, completeness, usefulness or suitability of any information contained herein. By reviewing this document, you agree that your use of any information contained herein is at your own risk, and you accept all risks and responsibility for losses, damages, costs and other consequences resulting directly or indirectly from any information or material available on it. Tolly is not responsible for, and you agree to hold Tolly and its related affiliates harmless from any loss, harm, injury or damage resulting from or arising out of your use of or reliance on any of the information provided herein. Tolly makes no claim as to whether any product or company described herein is suitable for investment. You should obtain your own independent professional advice, whether legal, accounting or otherwise, before proceeding with any investment or project related to any information, products or companies described herein. When foreign translations exist, the English document is considered authoritative. To assure accuracy, only use documents downloaded directly from Tolly.com. No part of any document may be reproduced, in whole or in part, without the specific written permission of Tolly. All trademarks used in the document are owned by their respective owners. You agree not to use any trademark in or as the whole or part of your own trademarks in connection with any activities, products or services which are not ours, or in a manner which may be confusing, misleading or deceptive or in a manner that disparages us or our information, projects or developments. 212148 jwft1 jt-mts-wt-2012-11-29-VerO© 2012 Tolly Enterprises, LLC Tolly.com Page 6 of 6