iSCSI and CLEAR-Flow

1. Extreme Networks White Paper
© 2011 Extreme Networks, Inc. All rights reserved. Do not reproduce.
Make Your Network Mobile
Abstract
This white paper is targeted to enterprise and service pro-
vider IT managers who may be evaluating the ability of iSCSI
SANs to satisfy the performance requirements of their most
demanding storage applications. Storage application perfor-
mance is dependent on more than just the performance of
the storage array. The full data path and the integration with
the other layers of the data center architecture must be taken
into consideration. Optimizing the full data path from the
server, through the network to the storage can contribute to
significant improvements in performance and service levels.
This white paper and the accompanying configuration guide
details the work of Intel, Extreme Networks® and NetApp to
demonstrate the impact on iSCSI performance in real world
environments using Extreme Networks CLEAR-Flow technol-
ogy to help you achieve optimal iSCSI performance.
The output of the collaborative testing is presented in two
separate documents:
• White Paper: provides an overview of the latest advance-
ments around iSCSI SAN capabilities and performance,
including real world, end-to-end performance results
with CLEAR-Flow.
• Configuration Guide: provides a sample reference archi-
tecture with step-by-step configuration details for all
the components and describes the end-to-end solution
performance across a number of configurations.
High-Performance iSCSI
SAN Architecture—
A Solution’s View from
Extreme Networks, Intel
and NetApp

2. 2
Extreme Networks White Paper: High Performance iSCSI SAN Architecture—A Solution’s View from Extreme Networks, Intel and NetApp
© 2011 Extreme Networks, Inc. All rights reserved.
Executive Summary
Recent technological innovations are changing the
way data centers are engineered and operated. Three
technology trends driving new economics and perfor-
mance in the data center include server virtualization,
unified storage based upon Ethernet storage protocols,
and high-performance Ethernet networking.
Specifically, as related to storage infrastructure, we have
seen a significant increase in the market acceptance of
Ethernet-based storage solutions, such as iSCSI SANs
and NAS. IDC estimates that by 2014, Ethernet storage
revenue will surpass that of legacy Fibre Channel-based
solutions.1
These trends signal a significant change in
storage architectures to support business critical and
high-demand storage applications that have traditionally
been deployed using dedicated Fibre Channel networks
to satisfy the desired performance and quality of service
requirements.
Historically, Ethernet-based storage solutions have
not been deployed for many business critical storage
applications. Ethernet, by its very nature, is a best efforts
transport mechanism. This fact has deterred many data
center managers from deploying Ethernet storage in
favor of Fibre Channel-based solutions. Fibre Channel
was architected as a lossless networking protocol to
address the requirements of high-performance
storage networks. In order to increase application of
iSCSI in more business critical storage applications,
Extreme Networks has enabled lossless networking
technology at the switch to provide a robust and
cost-effective 1 GbE and 10 GbE solution with increased
performance and quality of service. These capabilities
work at the solution level, to provide 1 GbE and 10 GbE
raw performance with lossless quality of service capa-
bilities previously only found in Fibre Channel-based
storage systems. These benefits are delivered through a
technology called CLEAR-Flow.
The performance requirements for some data center
applications can be sufficiently met with 1 GbE network
speeds. However for those applications that require
greater bandwidth and reduced latency, 10 GbE prod-
ucts are also available. In fact, the adoption of 10 GbE
is rapidly increasing due to a number of factors. First,
the deployment of server virtualization significantly
increases the network I/O density on each server.
Instead of servicing the I/O load of only one applica-
tion, servers today are managing the I/O load of many
applications running on the same hardware. As a result,
network bandwidth requirements have increased at the
server and edge switches. Ten Gigabit Ethernet
can meet the increased I/O demands with fewer ports
and cables.
Adoption of 10 GbE in the data center will accelerate
as next generation server motherboards transition to
embedded 10 GbE LAN on motherboard (LOM) ports
and both LOM and NICs offer the addition of more cost-
effective 10GBase-T (10 GbE copper) interconnects.
1
Gens, Frank. IDC Predictions 2011: Welcome to the New Mainstream. IDC.
December 2010. (IDC #225878, Volume: 1)
Figure 1: Multiple GbE connections in typical virtualized servers.
Figure 2: Simplified server connectivity using 10 GbE.

3. 3
Extreme Networks White Paper: High Performance iSCSI SAN Architecture—A Solution’s View from Extreme Networks, Intel and NetApp
© 2011 Extreme Networks, Inc. All rights reserved.
Increasing the Performance of
iSCSI Traffic
Lower Cost, Easier to Administer,
Converged
iSCSI is popular because of its “built-in” stack, which
is basically a storage layer on top of a well understood
and ubiquitously implemented TCP/IP stack. The ease
of administration of the iSCSI stack is afforded largely
because it is included in all major server operating
systems. No special drivers are needed with use of a
standard Ethernet server adapter, which can reduce risk
of component incompatibility. Native iSCSI initiators
now support advanced adapter features and platform
enhancements that deliver high performance to satisfy
the demands of enterprise applications. Native iSCSI
allows for easier server provisioning, which can contrib-
ute to lower operational expenditures.
The performance of an iSCSI storage network is depen-
dent upon the host adapter, the storage system and the
network infrastructure. From the network perspective,
iSCSI traffic must be protected and prioritized from other
traffic flowing through the infrastructure. Previously, this
was accomplished by leveraging two separate networks
– one for storage (traditionally a Fibre Channel SAN)
and another for the LAN traffic. A new set of 10 GbE
standards designed to provide lossless characteristics and
improved Quality of Service (QoS) is Data Center Bridging
(DCB). Combined with the Fibre Channel over Ethernet
(FCoE) standard, customers can now converge traditional
FC traffic onto a shared 10 GbE network. Support for IP
traffic isolation using DCB is still under development, but
is forthcoming. However, alternatives are available today
that can offer improved performance for iSCSI traffic in
congested environments.
The Power of CLEAR-Flow
Extreme Networks has developed a simple yet powerful
approach to data center network storage convergence
without having to compromise protection or perfor-
mance guarantees. CLEAR-Flow deploys a dynamic
policy that automatically identifies iSCSI traffic on the
network. Once identified, the switch prioritizes the
iSCSI traffic and creates a lossless environment that
is protected from non-iSCSI traffic on the network.
The implementation of dynamic iSCSI provisioning on
the network switch is accomplished in two steps. The
first step implements the policy at a network level,
by installing a global policy file on each switch in the
network, which then uses CLEAR-Flow to identify iSCSI
traffic. The second step dynamically applies a group of
parameters, in an automated fashion, at the port level
once iSCSI traffic is detected.
Many networks today rely on QoS alone to resolve
congestion issues and prioritize different traffic types
on a per port basis. Extreme Networks can augment the
benefits of QoS with its CLEAR-Flow engine to process
iSCSI traffic identified on the network. CLEAR-Flow is
a feature that allows Extreme Networks switches to
make forwarding decisions based on traffic type. Instead
of simply looking at the source and destination of the
traffic and forwarding it along the appropriate Layer 2 or
Layer 3 path, CLEAR-Flow takes things a step further by
allowing network administrators to specify certain types
of traffic that require more attention. In this case, the
switch identifies traffic on source and destination TCP
ports 3260. Once certain criteria for this traffic are met
(100 iSCSI frames per second), the switch performs the
following three actions:
1. Leverages QoS to assign the iSCSI traffic to a high
priority queue.
2. Enables jumbo frames on the identified ports.
3. Submits a log entry indicating that iSCSI traffic has
been identified and protected.
Once the iSCSI traffic has been identified and prioritized,
it can be protected against other types of traffic on the
network, including broadcast storms and other traffic
flow types that might compromise the performance of
iSCSI. This is made possible by placing all lower priority
traffic into lower priority queues where they can be
prioritized effectively under congested network condi-
tions. In extreme situations, lower priority packets may
be completely discarded in favor of iSCSI traffic.
This CLEAR-Flow enabled policy is configured as an
access control list on the switch, which in this case
permits all traffic and places iSCSI traffic into a higher
priority queue.
If the iSCSI traffic were to stop at any time, the switch’s
CLEAR-Flow engine dynamically identifies that the
traffic has completed and removes the associated
dynamic access control list, allowing hands-off admin-
istration. The iSCSI targets and initiators can be moved
anywhere throughout the network infrastructure without
the need for further administration of the switch.
This simple, yet effective CLEAR-Flow policy provides
superior dynamic prioritization and protection for all
iSCSI traffic on the network. Furthermore, the policy can
be deployed across any number of connected devices
where the traffic will be identified on all ports, includ-
ing edge, uplink and core. This ensures that the traffic
is protected and prioritized along the entire path the
packets take through the network.

4. 4
Extreme Networks White Paper: High Performance iSCSI SAN Architecture—A Solution’s View from Extreme Networks, Intel and NetApp
© 2011 Extreme Networks, Inc. All rights reserved.
Test Configuration Overview
This test environment has four basic components. These
components include the storage system, the server(s),
Ethernet server adapters, and the network switches. The
purpose of the test is to demonstrate how consistent
performance for iSCSI traffic can be maintained even
during times of network congestion, so that storage traffic
is not impacted by other traffic on the network.
The test configuration shown in Figure 3 consists of
five Intel Xeon processor 5600 series-based servers
running Microsoft Windows Server 2008 R2. The servers
are equipped with Intel Ethernet Server Adapter X520
series 10 GbE NICs using the native iSCSI initiator in
Windows Server 2008 R2. These NICs are connected
to an Extreme Networks Summit® X650 data center
switch, which is in turn connected via a 10 GbE port to a
NetApp® FAS6280 storage system. To create contention,
a second Extreme Networks Summit X650 switch was
connected in line to the first switch, with the storage
system connected to the second switch. The connection
between the two switches is another 10 GbE port on
each switch. Two test servers were then connected to
each switch to create contention on the link between the
switches.
The hardware and software used to execute the func-
tional and performance testing consists of:
• Intel white box dual-processor server
–– Two Intel Xeon processor 5680 CPUs
• 6 cores/12 threads each
• 3.33 GHz clock speed
–– Intel Ethernet Server Adapter X520 series
10 GbE adapters
–– 24G Memory
–– 2x146G HDD RAID 0
–– Windows Server 2008 R2
• NetApp FAS6280 storage system
–– Data ONTAP® 8.0.1 operating system
–– 2T Flash Cache (intelligent read cache)
• Extreme Networks Summit X650 10 GbE Switch
–– ExtremeXOS® v. 12.5
• Microsoft Windows Server 2008 R2
–– Native OS FCoE and iSCSI initiators
• Iometer version 2006.07.27
Test Results
The test results show that CLEAR-Flow clearly helps to
protect and prioritize iSCSI traffic. Each graph shows
four basic tests performed using Iometer.
1. No Contention with CLEAR-Flow – Shows what
would happen on a normal link with no other traffic
on the network.
2. Contention with CLEAR-Flow – In this case, there
is contention but CLEAR-Flow protects the iSCSI
traffic and ensures it is lossless.
3. No Contention and No CLEAR-Flow – The normal
case for iSCSI on a normal network, but a network
with no other traffic.
4. Contention and No CLEAR-Flow – The worst
case—no protection for the iSCSI traffic and
contention on the network. Throughput is
severely limited.
5582-01
W2K8
(iSCSI with Intel)
W2K8
(iSCSI with Intel)
W2K8
(iSCSI with Intel)
W2K8
(iSCSI with Intel)
W2K8
(iSCSI with Intel)
W2K8
(iSCSI with Intel)
W2K8
(iSCSI with Intel)
NetApp
FAS6280
Extreme Networks®
Summit® X650
Extreme Networks
Summit X650
10 GbE 10 GbE
10 GbE
10 GbE10 GbE
10 GbE
10 GbE
10 GbE
10 GbE
Figure 3: Test Configuration Architecture

5. 5
Extreme Networks White Paper: High Performance iSCSI SAN Architecture—A Solution’s View from Extreme Networks, Intel and NetApp
© 2011 Extreme Networks, Inc. All rights reserved.
The graphs are also divided between four different server configurations:
1. One server and one iSCSI session – a single server with a single stream
2. One server and eight iSCSI sessions – a single server with eight TCP streams
5583-01
IOmeter – 1 Server and 1 iSCSI Session/Server
(with/without CLEAR-Flow)
2,500
2,250
2,000
1,750
1,500
1,250
1,000
750
500
250
No Contention
CLEAR-Flow
Contention
CLEAR-Flow
No Contention
No CLEAR-Flow
Contention
No CLEAR-Flow
0
100
90
80
70
60
50
40
30
20
10
0
Throughput(MBps)
AverageCPUUtil(%)
512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB
MBps % CPU
5584-01
IOmeter – 1 Server and 8 iSCSI Session/Server
(with/without CLEAR-Flow)
2,500
2,250
2,000
1,750
1,500
1,250
1,000
750
500
250
No Contention
CLEAR-Flow
Contention
CLEAR-Flow
No Contention
No CLEAR-Flow
Contention
No CLEAR-Flow
0
100
90
80
70
60
50
40
30
20
10
0
Throughput(MBps)
AverageCPUUtil(%)
512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB
MBps % CPU

6. 6
Extreme Networks White Paper: High Performance iSCSI SAN Architecture—A Solution’s View from Extreme Networks, Intel and NetApp
© 2011 Extreme Networks, Inc. All rights reserved.
3. Five servers and one iSCSI session – five servers (enough to oversaturate the 10 GbE link) and a single stream
per server
4. Five servers and eight iSCSI sessions - five servers and eight streams, this being the most taxing test
5585-01
IOmeter – 5 Servers and 1 iSCSI Session/Server
(with/without CLEAR-Flow)
2,500
2,250
2,000
1,750
1,500
1,250
1,000
750
500
250
No Contention
CLEAR-Flow
Contention
CLEAR-Flow
No Contention
No CLEAR-Flow
Contention
No CLEAR-Flow
0
100
90
80
70
60
50
40
30
20
10
0
Throughput(MBps)
AverageCPUUtil(%)
512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB
MBps % CPU
5586-01
IOmeter – 5 Servers and 8 iSCSI Session/Server
(with/without CLEAR-Flow)
2,500
2,250
2,000
1,750
1,500
1,250
1,000
750
500
250
No Contention
CLEAR-Flow
Contention
CLEAR-Flow
No Contention
No CLEAR-Flow
Contention
No CLEAR-Flow
0
100
90
80
70
60
50
40
30
20
10
0
Throughput(MBps)
AverageCPUUtil(%)
512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB 512B 4KB 16KB
MBps % CPU

7. Make Your Network Mobile
Extreme Networks White Paper: Innovation With ExtremeXOS
© 2011 Extreme Networks, Inc. All rights reserved. Extreme Networks, the Extreme Networks Logo, ExtremeXOS and Summit are either registered
trademarks or trademarks of Extreme Networks, Inc. in the United States and/or other countries. NetApp, the NetApp logo, Go further, faster, and Data
ONTAP are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. All other trademarks are the trademarks of
their respective owners. Specifications are subject to change without notice. 1772_01 03/11
www.extremenetworks.com
Corporate
and North America
Extreme Networks, Inc.
3585 Monroe Street
Santa Clara, CA 95051 USA
Phone +1 408 579 2800
Europe, Middle East, Africa
and South America
Phone +31 30 800 5100
Asia Pacific
Phone +65 6836 5437
Japan
Phone +81 3 5842 4011
Conclusion
Market acceptance of Ethernet-based storage solutions,
such as iSCSI and NAS, is increasing as companies of
all sizes recognize economic and operational benefits
of converged data center networks based on Ethernet
technology. Trends, such as server virtualization and
service-based infrastructures, signal a significant
change in storage architectures to support business
critical and high-demand storage applications that
have traditionally been deployed using dedicated single
purpose networks.
Ethernet network throughput can be significantly
affected by the level of overall congestion in the network,
which can be a challenge for block-based storage
applications. By enabling CLEAR-Flow technology in
both 1 GbE and 10 GbE network environments, iSCSI
storage traffic can be protected, and offers lossless
storage performance along the data path. As servers
become more virtualized and network utilization rises,
this ability to prioritize data traffic is critical in order to
provide predictable iSCSI SAN performance.