SlideShare a Scribd company logo
1 of 10
Download to read offline
A Six-Step Plan for Competitive Device
         Evaluations
         How to Evaluate and Select the Best Content-Aware Network or Security Devices for Enterprise,
         Federal, and Carrier Infrastructures




BreakingPoint Enterprise IT Resiliency Series
www.breakingpoint.com
                                                                                                                                       1
© 2005 – 2011. BreakingPoint Systems, Inc. All rights reserved. The BreakingPoint logo is a trademark of BreakingPoint Systems, Inc.
All other trademarks are the property of their respective owners.
Executive Summary
IT organizations are upgrading to faster and more intelligent content-aware IT infrastructures to support customers, users,
and business operations as a whole. The sophisticated high-performance network and security devices within these
infrastructures require a more comprehensive approach to pre-deployment testing than traditional testing tools can provide.

This paper presents a six-step methodology for conducting competitive product evaluations that provide advance insight
into the performance, security, and stability of devices within production network and data center environments. By
following the methodology presented in this paper, organizations will:
  •	 Select the right firewall, IPS, UTM, load balancer, virtualized server, or other device to meet business and IT objectives.
  •	 Understand device capabilities to improve infrastructure planning and resiliency.
  •	 Save up to 50 percent on IT investments.
  •	 Eliminate hundreds of man-hours in post-purchase configuration, troubleshooting, and tuning.




                                                                                                                           2
Introduction
IT organizations embarking upon a network, security,               Recent high-profile performance and security failures are
or data center infrastructure upgrade need new                     bringing renewed focus to the importance of sufficient
methodologies and tools for testing and validating the             testing to ensure content-aware network devices can perform
performance, security, and stability of today’s content-           under real-world and peak conditions...Network equipment
aware devices. To make purchase decisions about firewalls,         providers, service providers, and other organizations require
intrusion prevention systems (IPS), servers, load balancers,       testing solutions capable of rigorously testing, simulating, and
and so on, CIOs, CISOs, and other IT leaders need better           emulating realistic application workloads and security attacks
information than traditional testing tools can provide.            at line speed.

Why? Because today’s content-aware and application-aware           Equally important, these tools must be able to keep pace with
devices employ deep packet inspection (DPI) capabilities to        emerging and more innovative products as well as thoroughly
examine traffic in ways that legacy testing approaches were
                                                                   vet complex content-aware/DPI-capable functionality by
never designed to validate. Such devices—and the complex
                                                                   emulating a myriad of application protocols and other types
traffic they handle—demand a new and deeper approach to
                                                                   of content at ever-increasing speeds to ensure delivery of an
comparative device testing that uses real application, attack,
                                                                   outstanding quality of experience (QoE) for the customer
and malformed traffic at ever-increasing speeds. Without
                                                                   and/or subscriber.
this improved approach, content-aware equipment cannot
be stressed thoroughly enough to determine the true limits
of its capabilities.                                               IDC Report: “The Inevitable Failure of Content-Aware/DPI
                                                                   Network Devices — and How to Mitigate the Risk”
This paper explains the six steps that organizations must
follow to validate DPI-enabled equipment and make fully
informed purchase decisions:                                       The providers of complex network security devices frequently
1. Create and prioritize specifications for products to be         make marketing claims that are unsupported by hard
    evaluated.
                                                                   evidence and, in any case, do not reflect the real-world
2. Develop a testing plan around repeatable, quantitative
                                                                   requirements of specific enterprises. The only solution for
    principles.
                                                                   prospective buyers is to define their own enterprise-specific
3. Use standardized scores to separate pretenders from
                                                                   business, security, and operational requirements, and test
    contenders.
                                                                   devices rigorously against those requirements...
4. Create individual test scenarios that mirror the
    production environment and are repeatable yet random.
5. Execute a layered testing progression that includes load,       Security professionals must be prepared to test in-line
    application traffic, security attacks, and other stress        security products to confirm their security effectiveness and
    vectors.                                                       performance capabilities under real-world conditions.
6. Lay the groundwork for successful deployment and
    maintenance.                                                   Gartner Report: “Guidelines for CISOs: A 10-Step Program for
                                                                   Selecting the Right Network Security Devices” (February 2011)
Why	Marketing	Claims	Are	Not	Sufficient
Vendor performance claims are based on generic conditions
within a vendor’s lab, and will never be sufficient for making   Test lab reports are equally inadequate. These labs follow
sound decisions. They can never accurately portray the           a “vacuum” or “clean room” approach, in which device
resiliency—the performance, security, and stability—of           testing is done in isolation, without regard to the unique
devices as they handle the unique mix of traffic within a        environments of customers. Also, test labs are often
customer’s network.                                              funded by device manufacturers, which invariably calls into
                                                                 question the objectivity of test results.



                                                                                                                                   3
Companies need an approach that allows them to impose           Six	Steps	to	the	Perfect	Competitive	Device	
their own conditions during pre-purchase evaluations—also       Evaluation
known as “bakeoffs”—so that they can rigorously validate        1.		 reate	and	Prioritize	Specifications	for	Products	to	Be	
                                                                   C
device capabilities under real-world scenarios at line rate.       Evaluated
Only by conducting this type of bakeoff will IT buyers          As with any project, it is wise to “begin with the end in
acquire the actionable answers needed to make informed          mind” when planning a device bakeoff. Before considering
purchase decisions and eliminate time-consuming post-           any piece of equipment, IT decision makers should clearly
deployment troubleshooting.                                     define and prioritize the organization’s needs for current and
                                                                future infrastructure build-out. Otherwise, it is too easy to
Benefits	of	a	Successful	Device	Evaluation                      dive into questions of speeds and feeds without taking into
IT professionals responsible for choosing content- or           account broader objectives. A good way to start is by asking
application-aware network and security equipment should         fundamental questions such as:
follow the steps outlined here to ensure that devices are         •	 How should the infrastructure support key business
fully evaluated before purchase and deployment. Using the            objectives? For example, what are the transaction
findings of pre-purchase and pre-deployment testing, they            latency requirements?
can refine their understanding of the unique real-world           •	 How important is the security of transactions in
conditions affecting their networks. By following the six            comparison to their speed?
steps for device bakeoffs explained in this paper, purchasers     •	 Which services are most sensitive, requiring the
will ensure that they:                                               highest levels of security?
  •	 Select	the	right	products	to	meet	their	business	            •	 Is application inspection necessary or not?
     objectives. Doing this requires a clear knowledge
     of device resiliency when handling a mix of real           The answers to questions like these may not be as obvious
     application traffic, security attacks, and malformed       as they initially appear. Obvious or not, they help establish
     traffic under heavy load.                                  the priorities for the infrastructure, which are then used to
  •	 Understand	device	capabilities	to	improve	                 generate specific evaluation criteria for selecting the right
     infrastructure	planning	and	resiliency. The                product.
     information gained during the bakeoff process allows
     IT planners to rightsize network and data center           Making this selection is about more than finding the right
     infrastructures to meet business needs for resiliency      make and model of device; it also means choosing the right
     while controlling costs.                                   amount of equipment to rightsize the infrastructure while
  •	 Save	up	to	50	percent	on	IT	investments. Customers         meeting business needs. To enable rightsizing, it is important
     that perform independent testing are able to rightsize
     infrastructures and pay for only the performance they
     actually get from each device. As this paper will show,      Testing is not necessarily about proving that the most-capable,
     buyers are negotiating better vendor discounts when          most-expensive product is the best choice. A well-designed
     armed with detailed information about the capabilities       testing plan may actually show that a lower level of performance
     of devices under their own network conditions.               is acceptable at certain points on the network, and this can
  •	 Eliminate	hundreds	of	man-hours	in	post-purchase	            reduce purchase and deployment costs. IT organizations that
     configuration	and	tuning. Performing thorough                do not perform relevant tests in-house may introduce serious
     pre-purchase validation gives purchasers advance             security and performance issues to their networks by purchasing
     knowledge of device capabilities and prevents weeks          underspecified devices, or may overspend significantly on higher
     of delays caused by post-deployment troubleshooting
                                                                  levels of performance and coverage that are not required.
     and vendor finger-pointing. This insight also helps IT
     organizations configure devices appropriately to avoid
                                                                  Gartner Report: “Protecting the Enterprise: Verifying the
     surprises and disruptions.
                                                                  Performance of Complex Network Security Products”
                                                                  (January 2011)



                                                                                                                                    4
to suspend assumptions about how many devices will be                 in Step 1 will translate into the specific parameters
required, because the approach to bakeoffs described here             evaluated during the testing itself.
often leads to surprising insights that overturn assumptions.
In one recent example, a large financial services organization        Elements	to	Include	in	the	Plan:
had planned to purchase two sets of redundant firewalls                •	 Controlled	Variables	— Throughout the planning
for a particular installation. But the company was working                and execution stages, the bakeoff must rigorously
from a false assumption about how many devices it would                   control variables. The goal should always be to isolate
actually need. A scientific bakeoff enabled the firewalls                 device capabilities and problem areas, which—as in
to be validated and properly tuned using the firm’s actual                any scientific investigation—requires repeating tests
network conditions instead of canned traffic or estimates.                exactly and changing only one input at a time.
This process revealed that the firewalls performed better              •	 Accurate	Baselines	— As a further control, each test
than expected. The company needed to buy only one set of                  process should include an initial run through a simple
redundant devices, not two, which cut its firewall bill in half.          piece of cable to establish what the traffic looks
                                                                          like without intermediation by any device. Doing
To facilitate the capture and rigorous analysis of                        so creates a valid basis for comparison against the
performance results, organizations may want to build out                  subsequent run of the same traffic through the DUT.
a Planning Matrix. In a spreadsheet, each important feature            •	 Uniform	Configurations	— Consistent collection
of devices to be evaluated is given its own row, with the                 of data requires that devices be set up uniformly
rows arranged in priority order. Weighted values are then                 throughout the bakeoff. This means that each vendor
assigned to each row. Each device being considered is given               should configure its device to match standard
its own column, and the columns are filled in as results are              settings established by the purchaser. It also means
gathered from the testing processes described below. When                 that settings for the testing equipment should be
the matrix is fully populated with performance details, it                maintained across all DUTs.
should provide objective clarity about how well each device            •	 Escalating	Complexity	— To achieve a comprehensive
performed across all criteria.                                            understanding of device capabilities, bakeoff testing
                                                                          should proceed by stages, evolving in complexity until
 Metric                  Weight   Device A    Device B    Device C        it fully reflects the purchaser’s unique mix of traffic. The
 Security Coverage
  Attacks Blocked (%)     35%        65          72          73
                                                                          first of these stages, explained in Step 3 below, uses
 Performance                                                              standards-based application, attack, and malformed
  Flows per Second        30%      111,374     97,764     119,384         traffic to evaluate a longer list of devices and quickly
  Max Concurrent Flows    20%     2,000,000   2,350,000   1,850,000
                                                                          eliminate obviously unsuitable choices. The second
  Throughput (Mbps)       15%      11,542      13,127      9,842
                                                                          stage, addressed in Steps 4 and 5, uses custom traffic
                                                                          mixes and progressive rounds of testing to precisely
                                                                          mirror the actual conditions that shortlisted devices
2.		 ethink	Testing	around	Repeatable,	Quantitative	
   R                                                                      will face once they are deployed in the purchaser’s
   Principles                                                             infrastructure.
Purchasers should use the specifications generated in                  •	 	 recision	Tools	— Bakeoffs must use testing tools
                                                                          P
Step 1 to create a plan for stressing each device under                   that create precise real-world network conditions
test (DUT) with real-world application, attack, and                       again and again and enable variables to be changed
malformed traffic under load. Doing so is not as simple as                one at a time. These tools must also capture exact
taking older, ad hoc approaches to testing and injecting                  measurements of device behavior to enable accurate
authentic traffic. The entire plan must embrace a scientific              comparisons among devices.
methodology to accurately validate the capabilities of
DPI-enabled devices, which means it must use repeatable               In the past, network and security professionals have lacked
experiments that yield clear, quantitative results. Only this         the precision tools necessary to enforce truly consistent,
approach ensures that the evaluation criteria established             scientific standards across their testing processes. That



                                                                                                                                 5
has hampered their ability to make decisions based                   from the scores, they can then choose the three or four
on hard quantitative data and forced them to make                    most suitable devices for deep, customized testing. Using
estimates about device resiliency based on whatever                  standardized scores at this stage saves time and money by
performance numbers they could gather. Today, however,               quickly establishing which devices are the most likely to fulfill
superior tools create authentic application traffic and              the business needs set out earlier in the process. Formulation
capture precise measurements of its effects, even for the            of the custom tests for shortlisted devices is covered in Step
complex interactions common in 10GigE content-aware                  4, while execution of them is addressed in Step 5.
environments. Companies that lack such tools can employ
them for the duration of a bakeoff by contracting with a             4.		 reate	Individual	Test	Scenarios	That	Mirror	the	
                                                                        C
third party for on-demand device evaluation services.                   Production	Environment	and	Are	Repeatable	yet	
                                                                        Random
3.		 se	Standardized	Scores	to	Separate	Pretenders	from	
   U                                                                 With this step, the bakeoff process moves into
   Contenders                                                        comprehensive testing to fully validate the capabilities
Purchasers can turn a long list of candidate devices into a          of DPI-enabled devices. Authentic validation requires an
short list without performing comprehensive validation on            accurate understanding of the application, network, and
each product by using standardized scoring methods. These            security landscape in which devices will be operating.
scores can quickly eliminate from consideration equipment            Therefore, purchasers should review their own traffic
that clearly does not meet an organization’s needs.                  mix and the mixes of service providers before designing
                                                                     individual tests; this will ensure that their testing equipment
For example, BreakingPoint has developed a Resiliency Score          reflects the latest versions and types of application traffic
that is calculated using industry standards from organizations       that traverse their network. They should also consult
such as US–CERT, IEEE, and the IETF, as well as standard sets of     independent security research as well as the findings of their
security strikes and real-world traffic mixes from the world’s       own in-house network or security operations centers for the
largest service providers. This scientific, repeatable process       latest information on security attacks, including malware
is designed to enable meaningful comparisons without                 and evasions. Companies that need help in collecting this
partiality to any vendor. It uses a battery of simulations to        information can turn to on-demand services that specialize
evaluate a DUT’s capabilities in terms of throughput, sessions,      in network security.
robustness in the face of corrupted traffic, and security. The
resulting score is presented as a numeric grade from 1 to 100.       It is important to note that packet captures (PCAPs) of
Devices may receive no score if they fail to pass traffic at any     network traffic are inadequate for this survey of the
point or degrade to an unacceptable performance level. The           landscape, since they attempt to substitute a tiny slice
Resiliency Score takes the guesswork and subjectivity out of         of real traffic for a steady flow of it. Modern application-
validation and allows administrators to quickly understand           aware devices typically come equipped with huge cache
the degree to which system security will be impacted under           memory, allowing them to ignore repetitive traffic such
load, attack, and real-world application traffic.                    as that found in PCAPs. Simplistic traffic such as plain UDP
                                                                     or HTTP packets, IMIX, or a blend of a few homogenous
The product certification firm Underwriters Laboratories has         protocols is likewise inadequate, because it does not reflect
recently announced a similar standard, UL 2825, that uses a          the complexities under which DUTs will operate once they
scientific evaluation system to validate network and security        are put into production. For these reasons, real stateful
equipment. Once the standard is implemented, it will serve           application traffic, along with live attacks and malformed
as a vendor-neutral benchmark for the performance, security,         traffic, must be used to push devices to their limits.
and stability of devices. UL intends to publish certifications for
all equipment that meets the standards set forth in UL 2825.         Generating real stateful traffic, however, is not enough:
                                                                     Validation processes must also be repeatable yet random.
Purchasers can use one of these standardized scores to               Repeatability demands that the testing equipment generate
evaluate a list of perhaps six to 10 candidate devices. Working      the same traffic in the same way for each DUT to ensure



                                                                                                                                 6
accurate “apples to apples” comparisons. Randomization               processes described in this section ensure that the device
makes test traffic behave like real-world traffic, creating          being evaluated can easily handle the load it will face, in
unexpected patterns that force DUTs to work harder.                  terms of both sessions and application throughput. If the
Randomization prevents vendors from relying on self-                 device cannot pass these tests with traffic known to be
published performance numbers achieved in sterile                    free of attacks, there is no way it will process enough traffic
environments designed to show their wares in a favorable             once its security features are turned on or when it must also
light. Creating repeatable yet random traffic requires the           handle other stress vectors such as malformed traffic.
use of a pseudo-random number generator (PRNG). Using
a PRNG, the purchaser sets a seed value, which the testing             Sessions
equipment uses to create standardized tests by generating              This set of tests uses TCP traffic to validate the DUT’s ability
all data variants in the same way for each test executed,              to (1) create and tear down TCP sessions at a prescribed
whether for a single DUT or several.                                   rate and (2) handle a prescribed maximum number of
                                                                       concurrent sessions. Each of these tests can be run in stair-
Creating test scenarios around these guidelines reinforces             step fashion, ramping up the degree of stress by steady
the quantitative, scientific principles laid down in Step 2 and        increments until the device fails. This will determine
prepares the way for the actual battery of customized tests            whether the device achieves its advertised limits and how
to be performed in Step 5.                                             much headroom it has to handle peak traffic.

5.		 xecute	a	Layered	Testing	Progression	That	Includes	
   E                                                                   Application Traffic
   Load,	Application	Traffic,	Security	Attacks,	and	Other	             These tests determine a device’s ability to handle
   Stress	Vectors                                                      real stateful application traffic at high levels of load.
This stage is the “main event” of a competitive device                 BreakingPoint, for example, offers a standard Enterprise
evaluation and, as such, deserves more detailed treatment              application traffic mix that includes more than a dozen
here. During this stage, the wisdom of a progressive,                  of the protocols most commonly found traversing
scientific approach to testing will become clear. By changing          Global 2000 corporate networks. That mix can then
only one variable at a time and testing the parameters set             be customized by changing the weighting of various
forth at earlier stages, this progression will reveal the specific     protocols or by adding other protocols that better reflect
strengths and weaknesses of each product, replacing                    the customer’s unique network environment.
guesswork or uncertainty with verifiable results.
                                                                     The session and application traffic processes should all be
Once deployed, a device will not be subjected to one type of         run three times. The first pass is a baseline run, using only
stress at a time; instead, it must deal with application traffic,    a piece of cable and no DUT. The second pass is performed
heavy user load, security attacks, and malformed traffic all         with the DUT in place but with no security or inspection
at once. That is why the ultimate test in this progression will      policies turned on. This should result in the purest measure
bring together all of those elements into a single battery of        of the DUT’s maximum ability to relay traffic. The third pass
tests. But to develop a proper understanding of how a DUT            is performed with the device’s default security or inspection
handles specific types of stress, its ability to handle load and     policies turned on. Since the device will be handling traffic
attacks will be tested separately first. Subsequent processes        that includes no attacks, evasions, or malformed packets,
will combine validation of load, security, and stress vectors.       the policies should yield no positive results. But running this
During the bakeoff, customers should archive all of these            process will indicate the basic impact on performance that
tests so that they can be repeated exactly during the                comes from having the target device’s application-aware
deployment phase explained in Step 6.                                features engaged.

Load                                                                 Security
A device’s specialized capabilities—to block malicious traffic,      Having probed the DUT’s ability to handle load without
detect trigger keywords, and so on—are not meaningful                the complications of security attacks, it is time to try the
unless they perform adequately under heavy load. The                 opposite case: security without load. A firewall, IPS, or


                                                                                                                                    7
unified threat management (UTM) device will never be                   All	Stress	Vectors
better at blocking attacks than when it has no background              The layering process concludes by adding other stress
traffic to contend with, so this portion of the testing will           vectors that the DUT will encounter in a production
reveal how a DUT’s security features perform under ideal               environment.
conditions.
                                                                         Malformed Traffic
Keeping the device’s default security policies in place, the             This traffic can appear maliciously, or simply from device
customer runs a standard list of security attacks to see how             malfunction. Either way, malformed traffic is a fact of life
well the DUT catches known malicious traffic. The purchaser              on every network and must be included in the bakeoff
then customizes the tests in two ways: (1) tailoring the                 plan. This portion of the bakeoff progressively determines
strike list to exercise particular security policies within the          how a DUT responds to malformed traffic, including frame
device and then (2) tailoring the device’s security policies to          impairments at Layer 2, session fuzzing at Layers 3 and 4,
handle particular strikes relevant to the customer’s network             and application fuzzing at Layer 7.
environment. As with all of the other processes in the
bakeoff, these variables should be changed one at a time so              Evasions
that each test run can be used to isolate particular device              At a minimum, this part of the bakeoff should include TCP
capabilities and problem areas.                                          segmentation and IP fragmentation evasions. Depending
                                                                         on the customer’s network conditions, custom lists of
Besides establishing the basic security capabilities of a                evasions can be included as well.
firewall, IPS, or UTM, the customization in this portion of the
bakeoff will also give IT staff members an idea of what level          Adding these stress vectors to load and attacks completes
of support they can expect from a manufacturer. Vendors                the picture. Performing a bakeoff in this way ensures that
will likely never be more responsive than when they are                the device being considered can cope with the entire set of
trying to close a sale, so the customer support during this            challenges it will face when deployed in the real world.
phase should be excellent.
                                                                       6.		 ay	the	Groundwork	for	Successful	Negotiation,	
                                                                          L
Combining	Load	and	Security                                               Deployment,	and	Maintenance
This phase of the bakeoff combines the ultimate tests from             Deploying untested network and security devices creates
the preceding Load and Security sections. While this does              serious problems for IT professionals, network and data
not complete the range of authentic conditions that will               center infrastructures, and organizations as a whole.
be included in the next testing phase, bringing these two              Untested equipment requires weeks of post-deployment
validation processes together may be a watershed for some              troubleshooting that is frustrating and time-consuming for
devices that simply cannot handle the combination of load              staff members and that often leads to both finger-pointing
and security attacks.                                                  and costly remediation steps. This is particularly true when
                                                                       device outages, security breaches, or unplanned bottlenecks
                                                                       impact the resiliency of entire infrastructures; such failures
  Evaluate shortlisted network security devices against a              damage reputation and business value while leading to
  realistic range of potential live attacks...
                                                                       serious, even career-limiting, embarrassment for individuals.
                                                                       By contrast, conducting a rigorous bakeoff minimizes the
  Testing can expose performance-related problems caused by
                                                                       risk of all these problems and saves hundreds of hours of
  inappropriate security products, including high latency and
                                                                       staff time by eliminating surprises and guesswork.
  frequent “fail closed events.” This, in turn, may result in active
  devices being deployed passively or blocking being disabled,
                                                                       Bakeoffs can also lower equipment prices. Before a purchase
  making the devices significantly less effective.
                                                                       is completed, customers should use the information
                                                                       generated during the bakeoff to negotiate a discount with
  Gartner Report: “Guidelines for CISOs”                               the chosen vendor. That information demonstrates the actual



                                                                                                                                8
Do not limit these testing procedures to the purchasing cycle    lab or disaster recovery backup environment. Beyond that,
  alone; make them an integral part of the ongoing security        pre-deployment testing enables predictive modeling across
  maintenance regime by implementing a solid, continuous
                                                                   a range of use-case scenarios, allowing the IT professional
                                                                   to understand how devices and infrastructures will perform
  testing initiative.
                                                                   under different configurations and network conditions. It
                                                                   also enables customers to hold vendors accountable for
  Gartner Report: “Guidelines for CISOs”
                                                                   supporting and improving their products over time.
                                                                   Ultimately, the benefits of pre-deployment testing extend
capabilities of the device under the customer’s own network        to the entire organization. Bakeoffs help IT organizations
conditions—not in the vendor’s lab. The purchaser can use          control costs and reduce risks while optimizing the
that data to argue for what the device should cost based on        performance, security, and stability of each device. Proper
demonstrated performance rather than marketing claims.             pre-deployment testing enables an organization to meet
For example, a company might select a firewall that meets          the key objectives outlined in Step 1 to deliver higher value
the specifications established in Step 1 of the bakeoff process    and meet business objectives. This approach mitigates the
but that blocks attacks at only 70 percent of its advertised       risks of outages and vulnerabilities, promotes rightsizing,
top speed. In that instance, it would be much easier for the       and drastically reduces the time, money, and frustration
customer to make the case that the vendor should offer a 30        required to deploy new devices.
percent discount on the price of the firewall.
                                                                   Summary
Once a device is purchased, the tests archived during Step 5       Purchasers of network and security devices should follow
should all be run again to enable proper configuration and         the scientific, quantitative progression of testing described
ensure that the device is production-ready. The detailed           here to fulfill the unique needs of their network and data
information created by these tests gives customers the             center infrastructures. Without following this approach,
advance insight needed to configure equipment to                   they will be unable to accurately assess the DPI capabilities
remediate weaknesses and achieve the optimal balance               of today’s content-aware devices operating in 10GigE
between performance and security. Using real-world traffic         environments. It is particularly important that they clearly
to tune the device also promotes rightsizing, because it           define necessary device specifications, use standard testing
allows engineers to build in enough of a performance               methodologies, and validate devices against network
cushion to handle peak traffic, but without creating waste         conditions that mirror reality. CISOs and other IT leaders can
by overbuilding that cushion. The exact data collected             follow the technical recommendations laid out in this paper,
from pre-deployment tests also makes it easier to work             or they can outsource the work to testing experts using on-
with vendors to remediate problems. Customer engineers             demand professional services.
can share definitive test results, forestalling arguments and
allowing vendors to correct problems more quickly. All of
these benefits enable staff members to deploy equipment            About	BreakingPoint
smoothly, without wasting time and money on remediating            BreakingPoint provides the turnkey services that
problems after the fact.                                           organizations need to gain advance insight into how
                                                                   devices, networks, and data centers will perform under their
The benefits of pre-deployment testing extend to entire            unique traffic mixes. BreakingPoint professional services
infrastructures as well. Advance simulation with real-world        provide actionable results in only days by leveraging the
conditions gives IT staff visibility into how device deployment    company’s patented product, dedicated security research
will impact other infrastructure elements and how those            team, and best practices from the Global 2000. Unlike any
elements will affect the device. These insights allow companies    other offering, these services enable IT professionals to
to install a new device with confidence that it will not disrupt   create the real-world simulations required to quickly and
the production environment, but without requiring the              cost-effectively harden IT resiliency, minimize IT risk, and
trouble and expense of deploying the device first in a test        train their own cyber warriors.



                                                                                                                            9
BreakingPoint	Device	Evaluation	Service
     The BreakingPoint Device Evaluation Service provides a                                       Contact BreakingPoint
     complete comparative evaluation, or “bakeoff,” of content-                                   Learn more about BreakingPoint products and services
     aware network, security, and data center devices using a                                     by contacting a representative in your area.
     customer’s own network conditions. The service includes                                      1.866.352.6691 U.S. Toll Free
     the setup and execution of high-performance stateful                                         www.breakingpoint.com
     application and attack simulations that mirror real-world
     traffic for each device. In less than a week, the customer                                   BreakingPoint Global Headquarters
     will receive detailed analysis of the performance, stability,                                3900 North Capital of Texas Highway
     and security of devices such as application servers, load                                    Austin, TX 78746
     balancers, firewalls, IDS/IPS devices, virus and spam filters,                               email: salesinfo@breakingpoint.com
     and more. A BreakingPoint Device Evaluation can be                                           tel: 512.821.6000
     conducted as a one-time project, providing the advance                                       toll free: 866.352.6691
     insight needed to confidently benchmark, select, and
     negotiate the purchase of IT devices. Or a yearly subscription                               BreakingPoint Government Solutions Group
     service includes ongoing Device Evaluations to ensure that                                   Washington, D.C.
     equipment remains resilient over time.                                                       email: government@breakingpoint.com
                                                                                                  tel: 703.443.1501
     Further	Reading
     BreakingPoint has published a series of step-by-step test                                    BreakingPoint EMEA Sales Office
     methodologies to help enterprises understand how to                                          Paris, France
     validate the performance, security, and stability of devices,                                email: emea_sales@breakingpoint.com
     networks, and data centers. Methodologies include:                                           tel: + 33 6 08 40 43 93
      •	 How to Test Dual Stack IPv4/IPv6
      •	 How to Test Server Load                                                                  BreakingPoint APAC Sales Office
      •	 How to Test Firewalls                                                                    Suite 2901, Building #5, Wanda Plaza
      •	 How to Test IPS Devices                                                                  No. 93 Jianguo Road
      •	 How to Test DPI Devices                                                                  Chaoyang District, Beijing, 100022, China
      •	 How to Test Server Load Balancers                                                        email: apac_sales@breakingpoint.com
                                                                                                  tel: + 86 10 5960 3162
     To access these resources, visit
     www.breakingpoint.com/services/.




www.breakingpoint.com
                                                                                                                                                    10
© 2005 – 2011. BreakingPoint Systems, Inc. All rights reserved. The BreakingPoint logo is a trademark of BreakingPoint Systems, Inc.
All other trademarks are the property of their respective owners.

More Related Content

What's hot

The Cloud: A game changer to test, at scale and in production, SOA based web...
The Cloud: A game changer to test, at scale and in production,  SOA based web...The Cloud: A game changer to test, at scale and in production,  SOA based web...
The Cloud: A game changer to test, at scale and in production, SOA based web...Fred Beringer
 
Use Your IDS Appliance, presented by Kate Brew, Product Marketing Manager at ...
Use Your IDS Appliance, presented by Kate Brew, Product Marketing Manager at ...Use Your IDS Appliance, presented by Kate Brew, Product Marketing Manager at ...
Use Your IDS Appliance, presented by Kate Brew, Product Marketing Manager at ...Ixia NVS Group
 
Capstone Presentation For Five Rivers Medical Centers
Capstone Presentation For Five Rivers Medical CentersCapstone Presentation For Five Rivers Medical Centers
Capstone Presentation For Five Rivers Medical Centersdjackson134
 
Advanced resource allocation and service level monitoring for container orche...
Advanced resource allocation and service level monitoring for container orche...Advanced resource allocation and service level monitoring for container orche...
Advanced resource allocation and service level monitoring for container orche...Conference Papers
 
2 new hw_features_cat_cod_etc
2 new hw_features_cat_cod_etc2 new hw_features_cat_cod_etc
2 new hw_features_cat_cod_etcvideos
 
Data Center Security
Data Center SecurityData Center Security
Data Center SecurityCisco Canada
 
Overcoming Scaling Challenges in MongoDB Deployments with SSD
Overcoming Scaling Challenges in MongoDB Deployments with SSDOvercoming Scaling Challenges in MongoDB Deployments with SSD
Overcoming Scaling Challenges in MongoDB Deployments with SSDMongoDB
 
Floorvision Brochure
Floorvision BrochureFloorvision Brochure
Floorvision BrochureFides Sales
 
Take the guesswork out of video surveillance with pre-validated Milestone s...
Take the guesswork out of video  surveillance with pre-validated  Milestone s...Take the guesswork out of video  surveillance with pre-validated  Milestone s...
Take the guesswork out of video surveillance with pre-validated Milestone s...Principled Technologies
 
Disrupting the Data Center: Unleashing the Digital Services Economy
Disrupting the Data Center: Unleashing the Digital Services EconomyDisrupting the Data Center: Unleashing the Digital Services Economy
Disrupting the Data Center: Unleashing the Digital Services EconomyIntel IT Center
 
Top 5 Reasons to Select SolarWinds over HP NNMi
Top 5 Reasons to Select SolarWinds over HP NNMi Top 5 Reasons to Select SolarWinds over HP NNMi
Top 5 Reasons to Select SolarWinds over HP NNMi SolarWinds
 
Knorr-Bremse Group Strong Authentication Case Study
Knorr-Bremse Group Strong Authentication Case StudyKnorr-Bremse Group Strong Authentication Case Study
Knorr-Bremse Group Strong Authentication Case StudySafeNet
 
Virtualizing More While Improving Risk Posture – From Bare Metal to End Point
Virtualizing More While Improving Risk Posture – From Bare Metal to End PointVirtualizing More While Improving Risk Posture – From Bare Metal to End Point
Virtualizing More While Improving Risk Posture – From Bare Metal to End PointHyTrust
 
Dell EMC validates your virtualized Genetec video management system before it...
Dell EMC validates your virtualized Genetec video management system before it...Dell EMC validates your virtualized Genetec video management system before it...
Dell EMC validates your virtualized Genetec video management system before it...Principled Technologies
 
Introduction to container networking in K8s - SDN/NFV London meetup
Introduction to container networking in K8s - SDN/NFV  London meetupIntroduction to container networking in K8s - SDN/NFV  London meetup
Introduction to container networking in K8s - SDN/NFV London meetupHaidee McMahon
 
IBM Software Defined Networking for Virtual Environments (IBM SDN VE)
IBM Software Defined Networking for Virtual Environments (IBM SDN VE)IBM Software Defined Networking for Virtual Environments (IBM SDN VE)
IBM Software Defined Networking for Virtual Environments (IBM SDN VE)IBM System Networking
 
NSA Capstone Presentation
NSA Capstone PresentationNSA Capstone Presentation
NSA Capstone PresentationMinh Vu
 

What's hot (20)

The Cloud: A game changer to test, at scale and in production, SOA based web...
The Cloud: A game changer to test, at scale and in production,  SOA based web...The Cloud: A game changer to test, at scale and in production,  SOA based web...
The Cloud: A game changer to test, at scale and in production, SOA based web...
 
Use Your IDS Appliance, presented by Kate Brew, Product Marketing Manager at ...
Use Your IDS Appliance, presented by Kate Brew, Product Marketing Manager at ...Use Your IDS Appliance, presented by Kate Brew, Product Marketing Manager at ...
Use Your IDS Appliance, presented by Kate Brew, Product Marketing Manager at ...
 
Capstone Presentation For Five Rivers Medical Centers
Capstone Presentation For Five Rivers Medical CentersCapstone Presentation For Five Rivers Medical Centers
Capstone Presentation For Five Rivers Medical Centers
 
Advanced resource allocation and service level monitoring for container orche...
Advanced resource allocation and service level monitoring for container orche...Advanced resource allocation and service level monitoring for container orche...
Advanced resource allocation and service level monitoring for container orche...
 
2 new hw_features_cat_cod_etc
2 new hw_features_cat_cod_etc2 new hw_features_cat_cod_etc
2 new hw_features_cat_cod_etc
 
Data Center Security
Data Center SecurityData Center Security
Data Center Security
 
Overcoming Scaling Challenges in MongoDB Deployments with SSD
Overcoming Scaling Challenges in MongoDB Deployments with SSDOvercoming Scaling Challenges in MongoDB Deployments with SSD
Overcoming Scaling Challenges in MongoDB Deployments with SSD
 
Floor Vision Brochure
Floor Vision BrochureFloor Vision Brochure
Floor Vision Brochure
 
Floorvision Brochure
Floorvision BrochureFloorvision Brochure
Floorvision Brochure
 
Take the guesswork out of video surveillance with pre-validated Milestone s...
Take the guesswork out of video  surveillance with pre-validated  Milestone s...Take the guesswork out of video  surveillance with pre-validated  Milestone s...
Take the guesswork out of video surveillance with pre-validated Milestone s...
 
Disrupting the Data Center: Unleashing the Digital Services Economy
Disrupting the Data Center: Unleashing the Digital Services EconomyDisrupting the Data Center: Unleashing the Digital Services Economy
Disrupting the Data Center: Unleashing the Digital Services Economy
 
Is the Network Tap Mightier Than the Sword
Is the Network Tap Mightier Than the SwordIs the Network Tap Mightier Than the Sword
Is the Network Tap Mightier Than the Sword
 
Net Optics' Virtualization Solutions Deployment Case Study
Net Optics' Virtualization Solutions Deployment Case StudyNet Optics' Virtualization Solutions Deployment Case Study
Net Optics' Virtualization Solutions Deployment Case Study
 
Top 5 Reasons to Select SolarWinds over HP NNMi
Top 5 Reasons to Select SolarWinds over HP NNMi Top 5 Reasons to Select SolarWinds over HP NNMi
Top 5 Reasons to Select SolarWinds over HP NNMi
 
Knorr-Bremse Group Strong Authentication Case Study
Knorr-Bremse Group Strong Authentication Case StudyKnorr-Bremse Group Strong Authentication Case Study
Knorr-Bremse Group Strong Authentication Case Study
 
Virtualizing More While Improving Risk Posture – From Bare Metal to End Point
Virtualizing More While Improving Risk Posture – From Bare Metal to End PointVirtualizing More While Improving Risk Posture – From Bare Metal to End Point
Virtualizing More While Improving Risk Posture – From Bare Metal to End Point
 
Dell EMC validates your virtualized Genetec video management system before it...
Dell EMC validates your virtualized Genetec video management system before it...Dell EMC validates your virtualized Genetec video management system before it...
Dell EMC validates your virtualized Genetec video management system before it...
 
Introduction to container networking in K8s - SDN/NFV London meetup
Introduction to container networking in K8s - SDN/NFV  London meetupIntroduction to container networking in K8s - SDN/NFV  London meetup
Introduction to container networking in K8s - SDN/NFV London meetup
 
IBM Software Defined Networking for Virtual Environments (IBM SDN VE)
IBM Software Defined Networking for Virtual Environments (IBM SDN VE)IBM Software Defined Networking for Virtual Environments (IBM SDN VE)
IBM Software Defined Networking for Virtual Environments (IBM SDN VE)
 
NSA Capstone Presentation
NSA Capstone PresentationNSA Capstone Presentation
NSA Capstone Presentation
 

Similar to White Paper: Six-Step Competitive Device Evaluation

Authentication_Best_Practices_WP(EN)_web
Authentication_Best_Practices_WP(EN)_webAuthentication_Best_Practices_WP(EN)_web
Authentication_Best_Practices_WP(EN)_webSafeNet
 
Extending the 20 critical security controls to gap assessments and security m...
Extending the 20 critical security controls to gap assessments and security m...Extending the 20 critical security controls to gap assessments and security m...
Extending the 20 critical security controls to gap assessments and security m...John M. Willis
 
Quality Assurance and its Importance in Software Industry by Aman Shukla
Quality Assurance and its Importance in Software Industry by Aman ShuklaQuality Assurance and its Importance in Software Industry by Aman Shukla
Quality Assurance and its Importance in Software Industry by Aman ShuklaAbhishekKumar773294
 
I Series User Management
I Series User ManagementI Series User Management
I Series User ManagementSJeffrey23
 
Education & Training39SecurityMagazine.com SECURITY JU
Education & Training39SecurityMagazine.com  SECURITY  JUEducation & Training39SecurityMagazine.com  SECURITY  JU
Education & Training39SecurityMagazine.com SECURITY JUEvonCanales257
 
Project Quality-SIPOCSelect a process of your choice and creat.docx
Project Quality-SIPOCSelect a process of your choice and creat.docxProject Quality-SIPOCSelect a process of your choice and creat.docx
Project Quality-SIPOCSelect a process of your choice and creat.docxwkyra78
 
Thick Client Penetration Testing Modern Approaches and Techniques.pdf
Thick Client Penetration Testing Modern Approaches and Techniques.pdfThick Client Penetration Testing Modern Approaches and Techniques.pdf
Thick Client Penetration Testing Modern Approaches and Techniques.pdfElanusTechnologies
 
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...kalichargn70th171
 
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...kalichargn70th171
 
Performance testing methodologies and tools
Performance testing methodologies and toolsPerformance testing methodologies and tools
Performance testing methodologies and toolsAlexander Decker
 
11.performance testing methodologies and tools
11.performance testing methodologies and tools11.performance testing methodologies and tools
11.performance testing methodologies and toolsAlexander Decker
 
NetWrix Change Reporter Suite - Product Review by Don Jones
NetWrix Change Reporter Suite - Product Review by Don JonesNetWrix Change Reporter Suite - Product Review by Don Jones
NetWrix Change Reporter Suite - Product Review by Don JonesNetwrix Corporation
 
selection of hardware & software in SAD
selection of hardware & software in SAD selection of hardware & software in SAD
selection of hardware & software in SAD Ankita Agrawal
 
Security Testing In The Secured World
Security Testing In The Secured WorldSecurity Testing In The Secured World
Security Testing In The Secured WorldJennifer Mary
 
Practical Enterprise Security Architecture
Practical Enterprise Security Architecture  Practical Enterprise Security Architecture
Practical Enterprise Security Architecture Priyanka Aash
 
Best Practices for Integrating Applications Development
Best Practices for Integrating Applications DevelopmentBest Practices for Integrating Applications Development
Best Practices for Integrating Applications DevelopmentKovair
 

Similar to White Paper: Six-Step Competitive Device Evaluation (20)

Authentication_Best_Practices_WP(EN)_web
Authentication_Best_Practices_WP(EN)_webAuthentication_Best_Practices_WP(EN)_web
Authentication_Best_Practices_WP(EN)_web
 
Extending the 20 critical security controls to gap assessments and security m...
Extending the 20 critical security controls to gap assessments and security m...Extending the 20 critical security controls to gap assessments and security m...
Extending the 20 critical security controls to gap assessments and security m...
 
Quality Assurance and its Importance in Software Industry by Aman Shukla
Quality Assurance and its Importance in Software Industry by Aman ShuklaQuality Assurance and its Importance in Software Industry by Aman Shukla
Quality Assurance and its Importance in Software Industry by Aman Shukla
 
I Series User Management
I Series User ManagementI Series User Management
I Series User Management
 
Education & Training39SecurityMagazine.com SECURITY JU
Education & Training39SecurityMagazine.com  SECURITY  JUEducation & Training39SecurityMagazine.com  SECURITY  JU
Education & Training39SecurityMagazine.com SECURITY JU
 
Computer system overview
Computer system overviewComputer system overview
Computer system overview
 
Project Quality-SIPOCSelect a process of your choice and creat.docx
Project Quality-SIPOCSelect a process of your choice and creat.docxProject Quality-SIPOCSelect a process of your choice and creat.docx
Project Quality-SIPOCSelect a process of your choice and creat.docx
 
Thick Client Penetration Testing Modern Approaches and Techniques.pdf
Thick Client Penetration Testing Modern Approaches and Techniques.pdfThick Client Penetration Testing Modern Approaches and Techniques.pdf
Thick Client Penetration Testing Modern Approaches and Techniques.pdf
 
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
 
Many products-no-security (1)
Many products-no-security (1)Many products-no-security (1)
Many products-no-security (1)
 
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
How Real Device Cloud Testing Ensures Exceptional Efficiency and Scalability ...
 
Performance testing methodologies and tools
Performance testing methodologies and toolsPerformance testing methodologies and tools
Performance testing methodologies and tools
 
11.performance testing methodologies and tools
11.performance testing methodologies and tools11.performance testing methodologies and tools
11.performance testing methodologies and tools
 
NetWrix Change Reporter Suite - Product Review by Don Jones
NetWrix Change Reporter Suite - Product Review by Don JonesNetWrix Change Reporter Suite - Product Review by Don Jones
NetWrix Change Reporter Suite - Product Review by Don Jones
 
selection of hardware & software in SAD
selection of hardware & software in SAD selection of hardware & software in SAD
selection of hardware & software in SAD
 
Security Testing In The Secured World
Security Testing In The Secured WorldSecurity Testing In The Secured World
Security Testing In The Secured World
 
Practical Enterprise Security Architecture
Practical Enterprise Security Architecture  Practical Enterprise Security Architecture
Practical Enterprise Security Architecture
 
Ta3s - Testing Banking and Finance Applications
Ta3s - Testing Banking and Finance ApplicationsTa3s - Testing Banking and Finance Applications
Ta3s - Testing Banking and Finance Applications
 
Best Practices for Integrating Applications Development
Best Practices for Integrating Applications DevelopmentBest Practices for Integrating Applications Development
Best Practices for Integrating Applications Development
 
Jon shende fbcs citp q&a
Jon shende fbcs citp q&aJon shende fbcs citp q&a
Jon shende fbcs citp q&a
 

More from Ixia

Measuring Private Cloud Resiliency
Measuring Private Cloud ResiliencyMeasuring Private Cloud Resiliency
Measuring Private Cloud ResiliencyIxia
 
LTE Testing
LTE TestingLTE Testing
LTE TestingIxia
 
BreakingPoint Storm CTM Cost-Effective Testing Solution
BreakingPoint Storm CTM Cost-Effective Testing SolutionBreakingPoint Storm CTM Cost-Effective Testing Solution
BreakingPoint Storm CTM Cost-Effective Testing SolutionIxia
 
BreakingPoint 3G Testing Data Sheet
BreakingPoint 3G Testing Data SheetBreakingPoint 3G Testing Data Sheet
BreakingPoint 3G Testing Data SheetIxia
 
LTE Testing | 4G Testing
LTE Testing | 4G TestingLTE Testing | 4G Testing
LTE Testing | 4G TestingIxia
 
IPS Test Methodology
IPS Test MethodologyIPS Test Methodology
IPS Test MethodologyIxia
 
Deep Packet Inspection (DPI) Test Methodology
Deep Packet Inspection (DPI) Test MethodologyDeep Packet Inspection (DPI) Test Methodology
Deep Packet Inspection (DPI) Test MethodologyIxia
 
How to Test High-Performance Next-Generation Firewalls
How to Test High-Performance Next-Generation FirewallsHow to Test High-Performance Next-Generation Firewalls
How to Test High-Performance Next-Generation FirewallsIxia
 
Catalyst 6500 ASA Service Module
Catalyst 6500 ASA Service ModuleCatalyst 6500 ASA Service Module
Catalyst 6500 ASA Service ModuleIxia
 
Testing High Performance Firewalls
Testing High Performance FirewallsTesting High Performance Firewalls
Testing High Performance FirewallsIxia
 
BreakingPoint & Fortinet RSA Conference 2011 Presentation: Evaluating Enterpr...
BreakingPoint & Fortinet RSA Conference 2011 Presentation: Evaluating Enterpr...BreakingPoint & Fortinet RSA Conference 2011 Presentation: Evaluating Enterpr...
BreakingPoint & Fortinet RSA Conference 2011 Presentation: Evaluating Enterpr...Ixia
 
BreakingPoint & Juniper RSA Conference 2011 Presentation: Securing the High P...
BreakingPoint & Juniper RSA Conference 2011 Presentation: Securing the High P...BreakingPoint & Juniper RSA Conference 2011 Presentation: Securing the High P...
BreakingPoint & Juniper RSA Conference 2011 Presentation: Securing the High P...Ixia
 
BreakingPoint & Stonesoft RSA Conference 2011 Presentation: Evaluating IPS
BreakingPoint & Stonesoft RSA Conference 2011 Presentation: Evaluating IPSBreakingPoint & Stonesoft RSA Conference 2011 Presentation: Evaluating IPS
BreakingPoint & Stonesoft RSA Conference 2011 Presentation: Evaluating IPSIxia
 
BreakingPoint & Juniper RSA Conference 2011 Presentation: Evaluating The Juni...
BreakingPoint & Juniper RSA Conference 2011 Presentation: Evaluating The Juni...BreakingPoint & Juniper RSA Conference 2011 Presentation: Evaluating The Juni...
BreakingPoint & Juniper RSA Conference 2011 Presentation: Evaluating The Juni...Ixia
 
BreakingPoint & Crossbeam RSA Conference 2011 Presentation: Evaluating High P...
BreakingPoint & Crossbeam RSA Conference 2011 Presentation: Evaluating High P...BreakingPoint & Crossbeam RSA Conference 2011 Presentation: Evaluating High P...
BreakingPoint & Crossbeam RSA Conference 2011 Presentation: Evaluating High P...Ixia
 
BreakingPoint & McAfee RSA Conference 2011 Presentation: Data Sheets Lie
BreakingPoint & McAfee RSA Conference 2011 Presentation: Data Sheets LieBreakingPoint & McAfee RSA Conference 2011 Presentation: Data Sheets Lie
BreakingPoint & McAfee RSA Conference 2011 Presentation: Data Sheets LieIxia
 
Cybersecurity: Arm and Train US Warriors to Win Cyber War
Cybersecurity: Arm and Train US Warriors to Win Cyber WarCybersecurity: Arm and Train US Warriors to Win Cyber War
Cybersecurity: Arm and Train US Warriors to Win Cyber WarIxia
 
Firewall Testing Methodology
Firewall Testing MethodologyFirewall Testing Methodology
Firewall Testing MethodologyIxia
 
BreakingPoint Cloud and Virtualization Data Sheet
BreakingPoint Cloud and Virtualization Data SheetBreakingPoint Cloud and Virtualization Data Sheet
BreakingPoint Cloud and Virtualization Data SheetIxia
 
Measure Network Performance, Security and Stability
Measure Network Performance, Security and StabilityMeasure Network Performance, Security and Stability
Measure Network Performance, Security and StabilityIxia
 

More from Ixia (20)

Measuring Private Cloud Resiliency
Measuring Private Cloud ResiliencyMeasuring Private Cloud Resiliency
Measuring Private Cloud Resiliency
 
LTE Testing
LTE TestingLTE Testing
LTE Testing
 
BreakingPoint Storm CTM Cost-Effective Testing Solution
BreakingPoint Storm CTM Cost-Effective Testing SolutionBreakingPoint Storm CTM Cost-Effective Testing Solution
BreakingPoint Storm CTM Cost-Effective Testing Solution
 
BreakingPoint 3G Testing Data Sheet
BreakingPoint 3G Testing Data SheetBreakingPoint 3G Testing Data Sheet
BreakingPoint 3G Testing Data Sheet
 
LTE Testing | 4G Testing
LTE Testing | 4G TestingLTE Testing | 4G Testing
LTE Testing | 4G Testing
 
IPS Test Methodology
IPS Test MethodologyIPS Test Methodology
IPS Test Methodology
 
Deep Packet Inspection (DPI) Test Methodology
Deep Packet Inspection (DPI) Test MethodologyDeep Packet Inspection (DPI) Test Methodology
Deep Packet Inspection (DPI) Test Methodology
 
How to Test High-Performance Next-Generation Firewalls
How to Test High-Performance Next-Generation FirewallsHow to Test High-Performance Next-Generation Firewalls
How to Test High-Performance Next-Generation Firewalls
 
Catalyst 6500 ASA Service Module
Catalyst 6500 ASA Service ModuleCatalyst 6500 ASA Service Module
Catalyst 6500 ASA Service Module
 
Testing High Performance Firewalls
Testing High Performance FirewallsTesting High Performance Firewalls
Testing High Performance Firewalls
 
BreakingPoint & Fortinet RSA Conference 2011 Presentation: Evaluating Enterpr...
BreakingPoint & Fortinet RSA Conference 2011 Presentation: Evaluating Enterpr...BreakingPoint & Fortinet RSA Conference 2011 Presentation: Evaluating Enterpr...
BreakingPoint & Fortinet RSA Conference 2011 Presentation: Evaluating Enterpr...
 
BreakingPoint & Juniper RSA Conference 2011 Presentation: Securing the High P...
BreakingPoint & Juniper RSA Conference 2011 Presentation: Securing the High P...BreakingPoint & Juniper RSA Conference 2011 Presentation: Securing the High P...
BreakingPoint & Juniper RSA Conference 2011 Presentation: Securing the High P...
 
BreakingPoint & Stonesoft RSA Conference 2011 Presentation: Evaluating IPS
BreakingPoint & Stonesoft RSA Conference 2011 Presentation: Evaluating IPSBreakingPoint & Stonesoft RSA Conference 2011 Presentation: Evaluating IPS
BreakingPoint & Stonesoft RSA Conference 2011 Presentation: Evaluating IPS
 
BreakingPoint & Juniper RSA Conference 2011 Presentation: Evaluating The Juni...
BreakingPoint & Juniper RSA Conference 2011 Presentation: Evaluating The Juni...BreakingPoint & Juniper RSA Conference 2011 Presentation: Evaluating The Juni...
BreakingPoint & Juniper RSA Conference 2011 Presentation: Evaluating The Juni...
 
BreakingPoint & Crossbeam RSA Conference 2011 Presentation: Evaluating High P...
BreakingPoint & Crossbeam RSA Conference 2011 Presentation: Evaluating High P...BreakingPoint & Crossbeam RSA Conference 2011 Presentation: Evaluating High P...
BreakingPoint & Crossbeam RSA Conference 2011 Presentation: Evaluating High P...
 
BreakingPoint & McAfee RSA Conference 2011 Presentation: Data Sheets Lie
BreakingPoint & McAfee RSA Conference 2011 Presentation: Data Sheets LieBreakingPoint & McAfee RSA Conference 2011 Presentation: Data Sheets Lie
BreakingPoint & McAfee RSA Conference 2011 Presentation: Data Sheets Lie
 
Cybersecurity: Arm and Train US Warriors to Win Cyber War
Cybersecurity: Arm and Train US Warriors to Win Cyber WarCybersecurity: Arm and Train US Warriors to Win Cyber War
Cybersecurity: Arm and Train US Warriors to Win Cyber War
 
Firewall Testing Methodology
Firewall Testing MethodologyFirewall Testing Methodology
Firewall Testing Methodology
 
BreakingPoint Cloud and Virtualization Data Sheet
BreakingPoint Cloud and Virtualization Data SheetBreakingPoint Cloud and Virtualization Data Sheet
BreakingPoint Cloud and Virtualization Data Sheet
 
Measure Network Performance, Security and Stability
Measure Network Performance, Security and StabilityMeasure Network Performance, Security and Stability
Measure Network Performance, Security and Stability
 

Recently uploaded

Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAndikSusilo4
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 

Recently uploaded (20)

Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 

White Paper: Six-Step Competitive Device Evaluation

  • 1. A Six-Step Plan for Competitive Device Evaluations How to Evaluate and Select the Best Content-Aware Network or Security Devices for Enterprise, Federal, and Carrier Infrastructures BreakingPoint Enterprise IT Resiliency Series www.breakingpoint.com 1 © 2005 – 2011. BreakingPoint Systems, Inc. All rights reserved. The BreakingPoint logo is a trademark of BreakingPoint Systems, Inc. All other trademarks are the property of their respective owners.
  • 2. Executive Summary IT organizations are upgrading to faster and more intelligent content-aware IT infrastructures to support customers, users, and business operations as a whole. The sophisticated high-performance network and security devices within these infrastructures require a more comprehensive approach to pre-deployment testing than traditional testing tools can provide. This paper presents a six-step methodology for conducting competitive product evaluations that provide advance insight into the performance, security, and stability of devices within production network and data center environments. By following the methodology presented in this paper, organizations will: • Select the right firewall, IPS, UTM, load balancer, virtualized server, or other device to meet business and IT objectives. • Understand device capabilities to improve infrastructure planning and resiliency. • Save up to 50 percent on IT investments. • Eliminate hundreds of man-hours in post-purchase configuration, troubleshooting, and tuning. 2
  • 3. Introduction IT organizations embarking upon a network, security, Recent high-profile performance and security failures are or data center infrastructure upgrade need new bringing renewed focus to the importance of sufficient methodologies and tools for testing and validating the testing to ensure content-aware network devices can perform performance, security, and stability of today’s content- under real-world and peak conditions...Network equipment aware devices. To make purchase decisions about firewalls, providers, service providers, and other organizations require intrusion prevention systems (IPS), servers, load balancers, testing solutions capable of rigorously testing, simulating, and and so on, CIOs, CISOs, and other IT leaders need better emulating realistic application workloads and security attacks information than traditional testing tools can provide. at line speed. Why? Because today’s content-aware and application-aware Equally important, these tools must be able to keep pace with devices employ deep packet inspection (DPI) capabilities to emerging and more innovative products as well as thoroughly examine traffic in ways that legacy testing approaches were vet complex content-aware/DPI-capable functionality by never designed to validate. Such devices—and the complex emulating a myriad of application protocols and other types traffic they handle—demand a new and deeper approach to of content at ever-increasing speeds to ensure delivery of an comparative device testing that uses real application, attack, outstanding quality of experience (QoE) for the customer and malformed traffic at ever-increasing speeds. Without and/or subscriber. this improved approach, content-aware equipment cannot be stressed thoroughly enough to determine the true limits of its capabilities. IDC Report: “The Inevitable Failure of Content-Aware/DPI Network Devices — and How to Mitigate the Risk” This paper explains the six steps that organizations must follow to validate DPI-enabled equipment and make fully informed purchase decisions: The providers of complex network security devices frequently 1. Create and prioritize specifications for products to be make marketing claims that are unsupported by hard evaluated. evidence and, in any case, do not reflect the real-world 2. Develop a testing plan around repeatable, quantitative requirements of specific enterprises. The only solution for principles. prospective buyers is to define their own enterprise-specific 3. Use standardized scores to separate pretenders from business, security, and operational requirements, and test contenders. devices rigorously against those requirements... 4. Create individual test scenarios that mirror the production environment and are repeatable yet random. 5. Execute a layered testing progression that includes load, Security professionals must be prepared to test in-line application traffic, security attacks, and other stress security products to confirm their security effectiveness and vectors. performance capabilities under real-world conditions. 6. Lay the groundwork for successful deployment and maintenance. Gartner Report: “Guidelines for CISOs: A 10-Step Program for Selecting the Right Network Security Devices” (February 2011) Why Marketing Claims Are Not Sufficient Vendor performance claims are based on generic conditions within a vendor’s lab, and will never be sufficient for making Test lab reports are equally inadequate. These labs follow sound decisions. They can never accurately portray the a “vacuum” or “clean room” approach, in which device resiliency—the performance, security, and stability—of testing is done in isolation, without regard to the unique devices as they handle the unique mix of traffic within a environments of customers. Also, test labs are often customer’s network. funded by device manufacturers, which invariably calls into question the objectivity of test results. 3
  • 4. Companies need an approach that allows them to impose Six Steps to the Perfect Competitive Device their own conditions during pre-purchase evaluations—also Evaluation known as “bakeoffs”—so that they can rigorously validate 1. reate and Prioritize Specifications for Products to Be C device capabilities under real-world scenarios at line rate. Evaluated Only by conducting this type of bakeoff will IT buyers As with any project, it is wise to “begin with the end in acquire the actionable answers needed to make informed mind” when planning a device bakeoff. Before considering purchase decisions and eliminate time-consuming post- any piece of equipment, IT decision makers should clearly deployment troubleshooting. define and prioritize the organization’s needs for current and future infrastructure build-out. Otherwise, it is too easy to Benefits of a Successful Device Evaluation dive into questions of speeds and feeds without taking into IT professionals responsible for choosing content- or account broader objectives. A good way to start is by asking application-aware network and security equipment should fundamental questions such as: follow the steps outlined here to ensure that devices are • How should the infrastructure support key business fully evaluated before purchase and deployment. Using the objectives? For example, what are the transaction findings of pre-purchase and pre-deployment testing, they latency requirements? can refine their understanding of the unique real-world • How important is the security of transactions in conditions affecting their networks. By following the six comparison to their speed? steps for device bakeoffs explained in this paper, purchasers • Which services are most sensitive, requiring the will ensure that they: highest levels of security? • Select the right products to meet their business • Is application inspection necessary or not? objectives. Doing this requires a clear knowledge of device resiliency when handling a mix of real The answers to questions like these may not be as obvious application traffic, security attacks, and malformed as they initially appear. Obvious or not, they help establish traffic under heavy load. the priorities for the infrastructure, which are then used to • Understand device capabilities to improve generate specific evaluation criteria for selecting the right infrastructure planning and resiliency. The product. information gained during the bakeoff process allows IT planners to rightsize network and data center Making this selection is about more than finding the right infrastructures to meet business needs for resiliency make and model of device; it also means choosing the right while controlling costs. amount of equipment to rightsize the infrastructure while • Save up to 50 percent on IT investments. Customers meeting business needs. To enable rightsizing, it is important that perform independent testing are able to rightsize infrastructures and pay for only the performance they actually get from each device. As this paper will show, Testing is not necessarily about proving that the most-capable, buyers are negotiating better vendor discounts when most-expensive product is the best choice. A well-designed armed with detailed information about the capabilities testing plan may actually show that a lower level of performance of devices under their own network conditions. is acceptable at certain points on the network, and this can • Eliminate hundreds of man-hours in post-purchase reduce purchase and deployment costs. IT organizations that configuration and tuning. Performing thorough do not perform relevant tests in-house may introduce serious pre-purchase validation gives purchasers advance security and performance issues to their networks by purchasing knowledge of device capabilities and prevents weeks underspecified devices, or may overspend significantly on higher of delays caused by post-deployment troubleshooting levels of performance and coverage that are not required. and vendor finger-pointing. This insight also helps IT organizations configure devices appropriately to avoid Gartner Report: “Protecting the Enterprise: Verifying the surprises and disruptions. Performance of Complex Network Security Products” (January 2011) 4
  • 5. to suspend assumptions about how many devices will be in Step 1 will translate into the specific parameters required, because the approach to bakeoffs described here evaluated during the testing itself. often leads to surprising insights that overturn assumptions. In one recent example, a large financial services organization Elements to Include in the Plan: had planned to purchase two sets of redundant firewalls • Controlled Variables — Throughout the planning for a particular installation. But the company was working and execution stages, the bakeoff must rigorously from a false assumption about how many devices it would control variables. The goal should always be to isolate actually need. A scientific bakeoff enabled the firewalls device capabilities and problem areas, which—as in to be validated and properly tuned using the firm’s actual any scientific investigation—requires repeating tests network conditions instead of canned traffic or estimates. exactly and changing only one input at a time. This process revealed that the firewalls performed better • Accurate Baselines — As a further control, each test than expected. The company needed to buy only one set of process should include an initial run through a simple redundant devices, not two, which cut its firewall bill in half. piece of cable to establish what the traffic looks like without intermediation by any device. Doing To facilitate the capture and rigorous analysis of so creates a valid basis for comparison against the performance results, organizations may want to build out subsequent run of the same traffic through the DUT. a Planning Matrix. In a spreadsheet, each important feature • Uniform Configurations — Consistent collection of devices to be evaluated is given its own row, with the of data requires that devices be set up uniformly rows arranged in priority order. Weighted values are then throughout the bakeoff. This means that each vendor assigned to each row. Each device being considered is given should configure its device to match standard its own column, and the columns are filled in as results are settings established by the purchaser. It also means gathered from the testing processes described below. When that settings for the testing equipment should be the matrix is fully populated with performance details, it maintained across all DUTs. should provide objective clarity about how well each device • Escalating Complexity — To achieve a comprehensive performed across all criteria. understanding of device capabilities, bakeoff testing should proceed by stages, evolving in complexity until Metric Weight Device A Device B Device C it fully reflects the purchaser’s unique mix of traffic. The Security Coverage Attacks Blocked (%) 35% 65 72 73 first of these stages, explained in Step 3 below, uses Performance standards-based application, attack, and malformed Flows per Second 30% 111,374 97,764 119,384 traffic to evaluate a longer list of devices and quickly Max Concurrent Flows 20% 2,000,000 2,350,000 1,850,000 eliminate obviously unsuitable choices. The second Throughput (Mbps) 15% 11,542 13,127 9,842 stage, addressed in Steps 4 and 5, uses custom traffic mixes and progressive rounds of testing to precisely mirror the actual conditions that shortlisted devices 2. ethink Testing around Repeatable, Quantitative R will face once they are deployed in the purchaser’s Principles infrastructure. Purchasers should use the specifications generated in • recision Tools — Bakeoffs must use testing tools P Step 1 to create a plan for stressing each device under that create precise real-world network conditions test (DUT) with real-world application, attack, and again and again and enable variables to be changed malformed traffic under load. Doing so is not as simple as one at a time. These tools must also capture exact taking older, ad hoc approaches to testing and injecting measurements of device behavior to enable accurate authentic traffic. The entire plan must embrace a scientific comparisons among devices. methodology to accurately validate the capabilities of DPI-enabled devices, which means it must use repeatable In the past, network and security professionals have lacked experiments that yield clear, quantitative results. Only this the precision tools necessary to enforce truly consistent, approach ensures that the evaluation criteria established scientific standards across their testing processes. That 5
  • 6. has hampered their ability to make decisions based from the scores, they can then choose the three or four on hard quantitative data and forced them to make most suitable devices for deep, customized testing. Using estimates about device resiliency based on whatever standardized scores at this stage saves time and money by performance numbers they could gather. Today, however, quickly establishing which devices are the most likely to fulfill superior tools create authentic application traffic and the business needs set out earlier in the process. Formulation capture precise measurements of its effects, even for the of the custom tests for shortlisted devices is covered in Step complex interactions common in 10GigE content-aware 4, while execution of them is addressed in Step 5. environments. Companies that lack such tools can employ them for the duration of a bakeoff by contracting with a 4. reate Individual Test Scenarios That Mirror the C third party for on-demand device evaluation services. Production Environment and Are Repeatable yet Random 3. se Standardized Scores to Separate Pretenders from U With this step, the bakeoff process moves into Contenders comprehensive testing to fully validate the capabilities Purchasers can turn a long list of candidate devices into a of DPI-enabled devices. Authentic validation requires an short list without performing comprehensive validation on accurate understanding of the application, network, and each product by using standardized scoring methods. These security landscape in which devices will be operating. scores can quickly eliminate from consideration equipment Therefore, purchasers should review their own traffic that clearly does not meet an organization’s needs. mix and the mixes of service providers before designing individual tests; this will ensure that their testing equipment For example, BreakingPoint has developed a Resiliency Score reflects the latest versions and types of application traffic that is calculated using industry standards from organizations that traverse their network. They should also consult such as US–CERT, IEEE, and the IETF, as well as standard sets of independent security research as well as the findings of their security strikes and real-world traffic mixes from the world’s own in-house network or security operations centers for the largest service providers. This scientific, repeatable process latest information on security attacks, including malware is designed to enable meaningful comparisons without and evasions. Companies that need help in collecting this partiality to any vendor. It uses a battery of simulations to information can turn to on-demand services that specialize evaluate a DUT’s capabilities in terms of throughput, sessions, in network security. robustness in the face of corrupted traffic, and security. The resulting score is presented as a numeric grade from 1 to 100. It is important to note that packet captures (PCAPs) of Devices may receive no score if they fail to pass traffic at any network traffic are inadequate for this survey of the point or degrade to an unacceptable performance level. The landscape, since they attempt to substitute a tiny slice Resiliency Score takes the guesswork and subjectivity out of of real traffic for a steady flow of it. Modern application- validation and allows administrators to quickly understand aware devices typically come equipped with huge cache the degree to which system security will be impacted under memory, allowing them to ignore repetitive traffic such load, attack, and real-world application traffic. as that found in PCAPs. Simplistic traffic such as plain UDP or HTTP packets, IMIX, or a blend of a few homogenous The product certification firm Underwriters Laboratories has protocols is likewise inadequate, because it does not reflect recently announced a similar standard, UL 2825, that uses a the complexities under which DUTs will operate once they scientific evaluation system to validate network and security are put into production. For these reasons, real stateful equipment. Once the standard is implemented, it will serve application traffic, along with live attacks and malformed as a vendor-neutral benchmark for the performance, security, traffic, must be used to push devices to their limits. and stability of devices. UL intends to publish certifications for all equipment that meets the standards set forth in UL 2825. Generating real stateful traffic, however, is not enough: Validation processes must also be repeatable yet random. Purchasers can use one of these standardized scores to Repeatability demands that the testing equipment generate evaluate a list of perhaps six to 10 candidate devices. Working the same traffic in the same way for each DUT to ensure 6
  • 7. accurate “apples to apples” comparisons. Randomization processes described in this section ensure that the device makes test traffic behave like real-world traffic, creating being evaluated can easily handle the load it will face, in unexpected patterns that force DUTs to work harder. terms of both sessions and application throughput. If the Randomization prevents vendors from relying on self- device cannot pass these tests with traffic known to be published performance numbers achieved in sterile free of attacks, there is no way it will process enough traffic environments designed to show their wares in a favorable once its security features are turned on or when it must also light. Creating repeatable yet random traffic requires the handle other stress vectors such as malformed traffic. use of a pseudo-random number generator (PRNG). Using a PRNG, the purchaser sets a seed value, which the testing Sessions equipment uses to create standardized tests by generating This set of tests uses TCP traffic to validate the DUT’s ability all data variants in the same way for each test executed, to (1) create and tear down TCP sessions at a prescribed whether for a single DUT or several. rate and (2) handle a prescribed maximum number of concurrent sessions. Each of these tests can be run in stair- Creating test scenarios around these guidelines reinforces step fashion, ramping up the degree of stress by steady the quantitative, scientific principles laid down in Step 2 and increments until the device fails. This will determine prepares the way for the actual battery of customized tests whether the device achieves its advertised limits and how to be performed in Step 5. much headroom it has to handle peak traffic. 5. xecute a Layered Testing Progression That Includes E Application Traffic Load, Application Traffic, Security Attacks, and Other These tests determine a device’s ability to handle Stress Vectors real stateful application traffic at high levels of load. This stage is the “main event” of a competitive device BreakingPoint, for example, offers a standard Enterprise evaluation and, as such, deserves more detailed treatment application traffic mix that includes more than a dozen here. During this stage, the wisdom of a progressive, of the protocols most commonly found traversing scientific approach to testing will become clear. By changing Global 2000 corporate networks. That mix can then only one variable at a time and testing the parameters set be customized by changing the weighting of various forth at earlier stages, this progression will reveal the specific protocols or by adding other protocols that better reflect strengths and weaknesses of each product, replacing the customer’s unique network environment. guesswork or uncertainty with verifiable results. The session and application traffic processes should all be Once deployed, a device will not be subjected to one type of run three times. The first pass is a baseline run, using only stress at a time; instead, it must deal with application traffic, a piece of cable and no DUT. The second pass is performed heavy user load, security attacks, and malformed traffic all with the DUT in place but with no security or inspection at once. That is why the ultimate test in this progression will policies turned on. This should result in the purest measure bring together all of those elements into a single battery of of the DUT’s maximum ability to relay traffic. The third pass tests. But to develop a proper understanding of how a DUT is performed with the device’s default security or inspection handles specific types of stress, its ability to handle load and policies turned on. Since the device will be handling traffic attacks will be tested separately first. Subsequent processes that includes no attacks, evasions, or malformed packets, will combine validation of load, security, and stress vectors. the policies should yield no positive results. But running this During the bakeoff, customers should archive all of these process will indicate the basic impact on performance that tests so that they can be repeated exactly during the comes from having the target device’s application-aware deployment phase explained in Step 6. features engaged. Load Security A device’s specialized capabilities—to block malicious traffic, Having probed the DUT’s ability to handle load without detect trigger keywords, and so on—are not meaningful the complications of security attacks, it is time to try the unless they perform adequately under heavy load. The opposite case: security without load. A firewall, IPS, or 7
  • 8. unified threat management (UTM) device will never be All Stress Vectors better at blocking attacks than when it has no background The layering process concludes by adding other stress traffic to contend with, so this portion of the testing will vectors that the DUT will encounter in a production reveal how a DUT’s security features perform under ideal environment. conditions. Malformed Traffic Keeping the device’s default security policies in place, the This traffic can appear maliciously, or simply from device customer runs a standard list of security attacks to see how malfunction. Either way, malformed traffic is a fact of life well the DUT catches known malicious traffic. The purchaser on every network and must be included in the bakeoff then customizes the tests in two ways: (1) tailoring the plan. This portion of the bakeoff progressively determines strike list to exercise particular security policies within the how a DUT responds to malformed traffic, including frame device and then (2) tailoring the device’s security policies to impairments at Layer 2, session fuzzing at Layers 3 and 4, handle particular strikes relevant to the customer’s network and application fuzzing at Layer 7. environment. As with all of the other processes in the bakeoff, these variables should be changed one at a time so Evasions that each test run can be used to isolate particular device At a minimum, this part of the bakeoff should include TCP capabilities and problem areas. segmentation and IP fragmentation evasions. Depending on the customer’s network conditions, custom lists of Besides establishing the basic security capabilities of a evasions can be included as well. firewall, IPS, or UTM, the customization in this portion of the bakeoff will also give IT staff members an idea of what level Adding these stress vectors to load and attacks completes of support they can expect from a manufacturer. Vendors the picture. Performing a bakeoff in this way ensures that will likely never be more responsive than when they are the device being considered can cope with the entire set of trying to close a sale, so the customer support during this challenges it will face when deployed in the real world. phase should be excellent. 6. ay the Groundwork for Successful Negotiation, L Combining Load and Security Deployment, and Maintenance This phase of the bakeoff combines the ultimate tests from Deploying untested network and security devices creates the preceding Load and Security sections. While this does serious problems for IT professionals, network and data not complete the range of authentic conditions that will center infrastructures, and organizations as a whole. be included in the next testing phase, bringing these two Untested equipment requires weeks of post-deployment validation processes together may be a watershed for some troubleshooting that is frustrating and time-consuming for devices that simply cannot handle the combination of load staff members and that often leads to both finger-pointing and security attacks. and costly remediation steps. This is particularly true when device outages, security breaches, or unplanned bottlenecks impact the resiliency of entire infrastructures; such failures Evaluate shortlisted network security devices against a damage reputation and business value while leading to realistic range of potential live attacks... serious, even career-limiting, embarrassment for individuals. By contrast, conducting a rigorous bakeoff minimizes the Testing can expose performance-related problems caused by risk of all these problems and saves hundreds of hours of inappropriate security products, including high latency and staff time by eliminating surprises and guesswork. frequent “fail closed events.” This, in turn, may result in active devices being deployed passively or blocking being disabled, Bakeoffs can also lower equipment prices. Before a purchase making the devices significantly less effective. is completed, customers should use the information generated during the bakeoff to negotiate a discount with Gartner Report: “Guidelines for CISOs” the chosen vendor. That information demonstrates the actual 8
  • 9. Do not limit these testing procedures to the purchasing cycle lab or disaster recovery backup environment. Beyond that, alone; make them an integral part of the ongoing security pre-deployment testing enables predictive modeling across maintenance regime by implementing a solid, continuous a range of use-case scenarios, allowing the IT professional to understand how devices and infrastructures will perform testing initiative. under different configurations and network conditions. It also enables customers to hold vendors accountable for Gartner Report: “Guidelines for CISOs” supporting and improving their products over time. Ultimately, the benefits of pre-deployment testing extend capabilities of the device under the customer’s own network to the entire organization. Bakeoffs help IT organizations conditions—not in the vendor’s lab. The purchaser can use control costs and reduce risks while optimizing the that data to argue for what the device should cost based on performance, security, and stability of each device. Proper demonstrated performance rather than marketing claims. pre-deployment testing enables an organization to meet For example, a company might select a firewall that meets the key objectives outlined in Step 1 to deliver higher value the specifications established in Step 1 of the bakeoff process and meet business objectives. This approach mitigates the but that blocks attacks at only 70 percent of its advertised risks of outages and vulnerabilities, promotes rightsizing, top speed. In that instance, it would be much easier for the and drastically reduces the time, money, and frustration customer to make the case that the vendor should offer a 30 required to deploy new devices. percent discount on the price of the firewall. Summary Once a device is purchased, the tests archived during Step 5 Purchasers of network and security devices should follow should all be run again to enable proper configuration and the scientific, quantitative progression of testing described ensure that the device is production-ready. The detailed here to fulfill the unique needs of their network and data information created by these tests gives customers the center infrastructures. Without following this approach, advance insight needed to configure equipment to they will be unable to accurately assess the DPI capabilities remediate weaknesses and achieve the optimal balance of today’s content-aware devices operating in 10GigE between performance and security. Using real-world traffic environments. It is particularly important that they clearly to tune the device also promotes rightsizing, because it define necessary device specifications, use standard testing allows engineers to build in enough of a performance methodologies, and validate devices against network cushion to handle peak traffic, but without creating waste conditions that mirror reality. CISOs and other IT leaders can by overbuilding that cushion. The exact data collected follow the technical recommendations laid out in this paper, from pre-deployment tests also makes it easier to work or they can outsource the work to testing experts using on- with vendors to remediate problems. Customer engineers demand professional services. can share definitive test results, forestalling arguments and allowing vendors to correct problems more quickly. All of these benefits enable staff members to deploy equipment About BreakingPoint smoothly, without wasting time and money on remediating BreakingPoint provides the turnkey services that problems after the fact. organizations need to gain advance insight into how devices, networks, and data centers will perform under their The benefits of pre-deployment testing extend to entire unique traffic mixes. BreakingPoint professional services infrastructures as well. Advance simulation with real-world provide actionable results in only days by leveraging the conditions gives IT staff visibility into how device deployment company’s patented product, dedicated security research will impact other infrastructure elements and how those team, and best practices from the Global 2000. Unlike any elements will affect the device. These insights allow companies other offering, these services enable IT professionals to to install a new device with confidence that it will not disrupt create the real-world simulations required to quickly and the production environment, but without requiring the cost-effectively harden IT resiliency, minimize IT risk, and trouble and expense of deploying the device first in a test train their own cyber warriors. 9
  • 10. BreakingPoint Device Evaluation Service The BreakingPoint Device Evaluation Service provides a Contact BreakingPoint complete comparative evaluation, or “bakeoff,” of content- Learn more about BreakingPoint products and services aware network, security, and data center devices using a by contacting a representative in your area. customer’s own network conditions. The service includes 1.866.352.6691 U.S. Toll Free the setup and execution of high-performance stateful www.breakingpoint.com application and attack simulations that mirror real-world traffic for each device. In less than a week, the customer BreakingPoint Global Headquarters will receive detailed analysis of the performance, stability, 3900 North Capital of Texas Highway and security of devices such as application servers, load Austin, TX 78746 balancers, firewalls, IDS/IPS devices, virus and spam filters, email: salesinfo@breakingpoint.com and more. A BreakingPoint Device Evaluation can be tel: 512.821.6000 conducted as a one-time project, providing the advance toll free: 866.352.6691 insight needed to confidently benchmark, select, and negotiate the purchase of IT devices. Or a yearly subscription BreakingPoint Government Solutions Group service includes ongoing Device Evaluations to ensure that Washington, D.C. equipment remains resilient over time. email: government@breakingpoint.com tel: 703.443.1501 Further Reading BreakingPoint has published a series of step-by-step test BreakingPoint EMEA Sales Office methodologies to help enterprises understand how to Paris, France validate the performance, security, and stability of devices, email: emea_sales@breakingpoint.com networks, and data centers. Methodologies include: tel: + 33 6 08 40 43 93 • How to Test Dual Stack IPv4/IPv6 • How to Test Server Load BreakingPoint APAC Sales Office • How to Test Firewalls Suite 2901, Building #5, Wanda Plaza • How to Test IPS Devices No. 93 Jianguo Road • How to Test DPI Devices Chaoyang District, Beijing, 100022, China • How to Test Server Load Balancers email: apac_sales@breakingpoint.com tel: + 86 10 5960 3162 To access these resources, visit www.breakingpoint.com/services/. www.breakingpoint.com 10 © 2005 – 2011. BreakingPoint Systems, Inc. All rights reserved. The BreakingPoint logo is a trademark of BreakingPoint Systems, Inc. All other trademarks are the property of their respective owners.