Cisco ASR 1000 Series Testing Results and Analysis

RA-070112-03 Cisco ASR 1000 Series Test Plan and Results July 2012

Temporary Text

Cisco ASR1000 Series
Router Test Plan and
Results

Report RA-070112-03
July 2012

1" Copyright © 2012 Router Analysis

RA-070112-03 Cisco ASR 1000 Series Test Plan and Results July 2012

Don’t Just Test. Analyze.
"

Quick Summary:
Cisc o ASR1004 Router The ASR1004 with ESP40 and RP2 Works As Expected
Test Plan and Re su lts When Configured With Features.
! Fabric Throughput Issues Do Not Impact QoS.
REP OR T R A-0 70 11 2 -0 3 Router Supports NetFlow On A Few Interfaces Without
Impacting Performance.

Devices covered in this report:
In this report Router Analysis tests the Cisco
Cisco: ASR1000 Series, specifically the ASR1004 along with the
ASR1004 RP2 / ESP40 ASR1006 in non-redundant mode.
ARR1006 RP2 / ESP40
With the latest RP2 processor and ESP40 Fabric,
How we test: how does the ASR1004 perform under stress?

Utilizing state-of-the-art Router Analysis has created and executed a test
testing technology from plan to confirm Cisco’s claims about performance, features
vendors such as IXIA and MU and QoS. We also check for head-of-line blocking issues.
"
Dynamics (now Spirent),
Router Analysis has a world In general we had no issues with the ASR and its
class testing lab. performance. The performance numbers mostly stayed
above the ones quoted by Cisco.

Read on to find out more.

2" Copyright © 2012 Router Analysis

RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012
!

Table of Contents

Table of Contents ................................................................................ 3!
Results Summary ................................................................................. 4!
Test Lab Setup .................................................................................... 5!
Tests Carried Out ................................................................................. 5!
Testing Results Reviewed .................................................................... 6!
Appendix A – Test Cases ................................................................... 17!
Appendix B – Test Result Data Tables ............................................... 21!
About Router Analysis ........................................................................ 22!

Table of Figures

Figure 1 - Cisco ASR1004 IPv4 and IPv6 Forwarding ........................... 4
Figure 2 – Cisco ASR1004 ................................................................... 5
Figure 3 - Cisco ASR1004 IPv4 and IPv6 Forwarding ........................... 6
Figure 4 - ACL + uRPF Results ............................................................ 8
Figure 5 - At < 50% Per Port We Saw No Issues ................................. 9
Figure 6 - QoS Loss On Tos 0 at 57% Line Rate .................................. 9
Figure 7 - QoS Loss on TOS 0 at 57% Line Rate ............................... 10
Figure 8 - At 69% TOS 4 is affected ................................................... 11
Figure 9 - 39% Loss On TOS 4 .......................................................... 11
Figure 10 - 88.52% Caused issues With All Streams .......................... 12
Figure 11 - Background Traffic ........................................................... 12
Figure 12 - Background Traffic 10GE ................................................. 13
Figure 13 - QoS + BG Traffic.............................................................. 13
Figure 14 - QoS + BG Traffic - Highlighted ......................................... 13
Figure 15 - HOLB Testing .................................................................. 14
Figure 16 - OSPF Neighbors .............................................................. 15
Figure 17 - RSVP Tunnels .................................................................. 15
Figure 18 - MPLS TE Traffic ............................................................... 16

3 Copyright © 2012 Router Analysis
!

!

Results Summary
The Cisco ASR1004 performed within Cisco’s claimed specifications when
tested at the Router Analysis test lab.

Featureless IPv4 forwarding performed better than advertised: forwarding at
27 Million packets per second (Mpps) versus the claimed 23Mpps.

Featureless IPv6 forwarding was tested at 21Mpps.

Enabling features such as ACLs and uRPF dropped the routers forwarding
performance down to 23.4Mpps for IPv4 and 11.7Mpps for IPv6. Both numbers
are above the 10.4Mpps minimum forwarding rate of the system advertised by
Cisco.

QoS functioned as expected, protecting streams as configured even with
fabric congestion. No head-of-line blocking was observed.

30000000

25000000

20000000

Claimed
15000000
Observed
w uRPF
10000000

5000000

0
IPv4 Base
IPv4 ACL
IPv6 Base
IPv6 ACL

Figure 1 - Cisco ASR1004 IPv4 and IPv6 Forwarding

!

!

!

Test Lab Setup
The Router Analysis test lab utilizes the latest in IXIA testing hardware. The
tests in this report were performed using an IXIA testing chassis with 22 Gigabit
Ethernet ports, two 10 Gigabit Ethernet ports along with IXIAs IxNetwork, IxLoad
and IxAutomate software.

The ASR1004 was configured with one RP2, one ESP40 and two SIP40
carrier cards. Each carrier card had three five port Gigabit Ethernet v2 SPAs and
one Single Port 10 Gigabit Ethernet v2 SPA.

Figure 2 – Cisco ASR1004

Tests Carried Out
• IP forwarding rate
o Minimum Packet Size (64 bytes)
! With ACLs (permit any any)
! With uRPF
! With QoS
! With NetFlow
o Packet IMIX + Features
o 1500 Byte Packets (Fabric Throughput) + Features
• Oversubscription
o One to Many (HOLB Test)
o Many to One
o QoS
• MPLS
o RSVP-TE Scale and Performance

!

!

Testing Results Reviewed
IPv4 and IPv6 Forwarding Results with Features

30000000

25000000

20000000

Claimed
15000000
Observed
w uRPF
10000000

5000000

0
IPv4 Base
IPv4 ACL
IPv6 Base
IPv6 ACL

Figure 3 - Cisco ASR1004 IPv4 and IPv6 Forwarding

The main takeaway from the results is that the ASR1004 performed above
the quoted specifications for featureless IPv4 based forwarding. IPv4 + uRPF also
came in above the stated 23Mpps specification. IPv4 forwarding with ACLs was
21.3Mpps (about the same as IPv6 base which is 21Mpps).

While testing the ASR1004, the lowest performance appeared while
executing the “IPv6 forwarding with ACLs, uRPF, QoS and NetFlow” test case. By
the time I had NetFlow configured on all of the interfaces the ASR1004
performance was down below 1Mpps.

Most if not all companies would never apply NetFlow inbound and outbound
on ALL interfaces. Running NetFlow on a few interfaces on the ASR1004 does not
cause any issues.

Both the IMIX and Fabric Bandwidth tests were above the minimum
threshold and able to be done at full fabric bandwidth (40G). The Fabric numbers
are uni-directional, if traffic is counted both directions we exceeded 80G across the
system.

!

!
IPv4/IPv6 Forwarding + Features Tests

Step 1: Configure 40G of ports to send traffic in a mesh

Result: No Loss Seen

Step 2: Configure a simple permit all ACL on the router:

Step 3: Configure the ACL inbound on one interface


Step 4: Configure the ACL outbound on one interface


Step 4: Configure the ACL inbound and outbound on all interfaces

Result: Traffic forwarding rate drops to 21Mpps

Step 5: Configure traffic to run at about 21Mpps

Step 6: Configure uRPF on one interface

Result: No Loss Seen (Hard to get the traffic perfect)

Step 7: Configure uRPF on a second interface

Result: Drop in performance of 70Kpps

Step 8: Configure uRPF on the rest of the interfaces checking the stats as
you go along.

Result: uRPF appears to cause a 25Kpps impact per GE port and a
250Kpps impact per 10GE port.

!

!

!
Figure 4 - ACL + uRPF Results

QoS Testing and Results:

From our tests we have determined that QoS works as expected on the
system. In a simple test involving three GbE links (a two to one oversubscription).
Three different TOS bits were set (0, 4 and 7) and traffic was sent balanced (33.3%
per TOS) from each GbE towards a single GbE port outbound.

We configured the router so that TOS 0 was protected up to 1% of line rate,
TOS 4 was protected up to 40% of line rate and TOS 7 was protected up to 70%
of line rate.

When running the test we saw the following results:

< 100% all streams went through without issue
> 100% TOS 0 started to be dropped
~ 140% TOS 4 started to be dropped
~ 190% TOS 7 started to be dropped

Step 1: Send less than 50% line-rate from each of the two ports

!

!

Figure 5 - At < 50% Per Port We Saw No Issues

Result: Up to almost 50% line-rate from the 2 ports (< 100% outbound on
other interface) we saw no loss. Above 50% we saw loss for the TOS 0 stream but
not the TOS 4 or 7 streams.

Step 2: Send 57% line-rate from each the two ports (114% total)

Figure 6 - QoS Loss On Tos 0 at 57% Line Rate

!

!
Result: At about 57% we start to see TOS 0 traffic drop as shown in the
magnified section below:

Figure 7 - QoS Loss on TOS 0 at 57% Line Rate

!

Step 3: Up the traffic to about 69% line-rate per port (138% total)

!

!

Figure 8 - At 69% TOS 4 is affected

As shown in this magnified section we see about 38% Loss on TOS 4 when
sending 69% line-rate per port (138% line-rate outbound)

Figure 9 - 39% Loss On TOS 4

!

!
Step 4: Raise the traffic to about 88.5% we start seeing traffic loss on all
streams.

Figure 10 - 88.52% Caused issues With All Streams

!
! Result: We see loss on all 3 QoS traffic profiles.

Final Result: From this basic test we have determined that QoS works as
expected.

!
QoS + Background Traffic + Fabric Loss/Forwarding Issues

We then did the same QoS test with background traffic between all other
ports on the system.

Figure 11 - Background Traffic

!

!

!
The background traffic saw packet loss as seen above and magnified below:

Figure 12 - Background Traffic 10GE

Here you can see we are sending 14Mpps and only getting 4Mpps on the
10GE ports.

And QoS traffic continued to function as expected:

Figure 13 - QoS + BG Traffic

Figure 14 - QoS + BG Traffic - Highlighted

As you can see there are no drops on the TOS 4 or TOS 7 traffic.
!

!
ASR1004 Policy Configuration

Here is the router configuration:

Policy Map ixia-test2
Class prec_0
bandwidth remaining ratio 10
police cir percent 1
conform-action transmit
exceed-action transmit
violate-action drop
Class prec_4
violate-action drop
Class prec_7

Head-of-Line Blocking Testing:

For HOLB Testing we setup a test case where we sent traffic from three GE
ports to one GE port, then from another GE port to both the over-loaded GE port
and an empty GE port. The red * indicates that the IXIA did not have real time
information at the time I took the screen shot.

Figure 15 - HOLB Testing

!
! !

!

!
MPLS Testing and Results

Step 1: Enable OSPF with MPLS Traffic Engineering
Cisco Example:
router ospf 1
mpls traffic-eng area 0

Step 2: Enable MPLS Traffic Engineering on the router and interfaces
Cisco Example
interface GigabitEthernet1/0/2
description to IXIA G3_2
ip address 198.18.8.1 255.255.255.0
ip ospf 1 area 0
mpls traffic-eng tunnels

Step 3: Configure the IXIA to do OSPF with the interfaces on the router

!
Figure 16 - OSPF Neighbors

Step 4: Configure the IXIA to generate 100 fully meshed tunnels on each
interface (1200 tunnels total).

!
Figure 17 - RSVP Tunnels

!

!

Step 5: Setup fully meshed traffic between all ports and run traffic to verify
functionality.

!
Figure 18 - MPLS TE Traffic

Step 6: Increase the number of tunnels until you can no longer forward
traffic, you see failures on the system or any functionality is compromised.

Result: Tested up to 4800 TE Tunnels without issue.
!

!

!

Appendix A – Test Cases
The following two test cases represent a skeleton format of the set of tests
performed in the Router Analysis lab.

Test Case One – IP Forwarding

Synopsis:

When testing systems which have a set PPS or Fabric Bandwidth which is lower
then the combined interfaces available; it is necessary to validate the best-case
base forwarding rate. Once the rate has been determined, it is possible to judge
other test cases against the best-case forwarding rate.

Goal of Test:

To confirm best-case IPv4 or IPv6 forwarding rate using the minimum and
maximum packet size accepted by the system.

Procedure:

Configure the minimum number of links between the tester and the system under
test (SUT) to provide either a.) More than the maximum PPS or b.) More than the
maximum fabric bandwidth as claimed by the vendor.

For example:

The ASR1004 ESP40 claims a maximum of 23Mpps or 40Gbps. To test 23Mpps,
you could use two 10GE interfaces running 64 byte packets at line rate (14.8Mpps
x 2).

To test the maximum fabric bandwidth you run large packets such as 1500 bytes
and would need at least 42G of interfaces such as four 10GE and two GE (due to
packet overhead which would be stripped before it reached the fabric).

Data to Collect:

Maximum PPS for IPv4, IPv6 and a 50/50 mix of IPv4 and IPv6
Maximum fabric bandwidth for IPv4, IPv6 and a 50/50 mix of IPv4 and IPv6

!

!

Test Case Two – IP Forwarding with Features

Synopsis:

Once the best-case forwarding rate for IP has been determined, it is important to
start enabling features to see what impact they may have on the forwarding rate.

Goal of Test:

To determine which features if any impact the forwarding rate of the SUT.

Procedure:

ACLs:

Using your best-case setup from test one, apply a simple “allow all” access list to
one interface inbound. Once you have recorded the data, swap the access list to
outbound on the same interface.

Next apply the access list both in and out on the same interface.

The final steps are to apply the access list in and out on the other interfaces
observing if the impact is the same or greater.

When you have completed the testing you should have the access list applied to all
interfaces inbound and outbound.

(If no impact is seen even with all interfaces running ACLs, you may need to
configure a bigger ACL using multiple Access List Entries (ACEs) with random
source/destination addresses and ports blocked before an allow all statement.)

If ACLs have no impact on forwarding performance it is possible that the SUT does
all access list actions in hardware and no impact will be seen. Consult with your
vendor to discuss.

uRPF:

For uRPF follow the same concept as with ACLs: apply uRPF to one interface at a
time, log the results and then continue.

!

!
NetFlow:

For NetFlow follow the same concept, a simple capture all NetFlow configuration
applied inbound and outbound
QoS:

For QoS, build a simple policer and add it to each interface inbound/outbound and
see if there is any impact.

Data to Collect:

Screen captures and if possible csv data for each of the tests you execute.
!

!

!
Test Case Three – QoS Testing

Synopsis:
!
This is a simple QoS test based on information from the IXIA BlackBook. We have
modified it slightly to utilize more than 2 ports.

Goal of Test:

Determine if QoS functions as expected by creating three different TOS valued
streams, 0, 4 and 7. On the router configure a traffic policy to prefer 7 over 4 and 4
over 0.

Procedure:

Using IxNetwork create three streams with TOS levels of 0, 4 and 7 on two
interfaces going towards a single interface.

On the outbound interface configure a QoS policy to prefer TOS 7 over TOS 4 and
TOS 4 over TOS 0.

Run traffic and slowly raise it until you see packets drop for 0, then 4 and then 7.

Additive Tests:

Add in background traffic from your earlier test where you saw packet loss due to
features. Confirm that the QoS policy is still working as expected.

Data to Collect:

Packet loss data for each stream at different traffic levels.
Screen Captures of the IxNetwork screen during the tests.

!

!

!

Appendix B – Test Result Data Tables
The following tables represent a sample of the data collected during testing in the
Router Analysis Lab.

IPv4 Forwarding Results Data

Test Claimed Test Results Pass/Fail
IPv4 PPS no ACL 23,000,000 26,600,000 Pass
IPv4 PPS with ACL 10,400,000 21,774,000 Pass
IPv4 Fabric BW 40G 40G Pass
IPv4 Fabric BW ACL 40G 40G Pass

IPv4 + uRPF Forwarding Results Data

Test Claimed Test Results Pass/Fail
IPv4 + uRPF 10,400,000 23,450,000 Pass
IPv4 ACL + uRPF 10,400,000 18,140,800 Pass
IPv4 Fabric + uRPF 40G 40G Pass
IPv4 Fabric ACL + uRPF 40G 40G Pass

IPv6 Forwarding Results Data

Test Claimed Test Result Pass/Fail
IPv6 PPS no ACL 10,400,000 21,000,000 Pass
IPv6 PPS with ACL 10,400,000 14,900,000 Pass
IPv6 Fabric BW 40G 40G Pass
IPv6 Fabric BW with ACL 40G 40G Pass

IPv6 + uRPF Forwarding Results Data

Test Claimed Test Result Pass/Fail
IPv6 + uRPF 10,400,000 17,500,000 Pass
IPv6 ACL + uRPF 10,400,000 14,900,000 Pass
IPv6 Fabric + uRPF 40G 40G Pass
IPv6 Fabric ACL + uRPF 40G 40G Pass

!

!

!

About Router Analysis
With a world-class fully vendor independent testing lab, Router Analysis
provides unbiased information to its clients.

Router Analysis provides the following services:

Router and Switch Testing
Test Plan Creation
On-Site Testing Support
Router, Switch and Tester Training
Network Design Consultation, Verification and Installation
… And More.

Contact Router Analysis by phone at +1 888 SONN COM (766-6266) or by
email at info@routeranalysis.com for a quote, more information or any questions.

!

Cisco ASR 1000 Series Testing Results and Analysis

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Viewers also liked

Viewers also liked (11)

Similar to Cisco ASR 1000 Series Testing Results and Analysis

Similar to Cisco ASR 1000 Series Testing Results and Analysis (20)

Recently uploaded

Recently uploaded (20)

Cisco ASR 1000 Series Testing Results and Analysis