Cisco ASR 1000 Series Testing Results and Analysis
Upcoming SlideShare
Loading in...5
×
 

Cisco ASR 1000 Series Testing Results and Analysis

on

  • 1,600 views

Router Analysis, Inc. tested the Cisco ASR1004 and 1006 routers with the latest RP2, ESP40 and SIP40 hardware. The router was outfitted with Six 5 port GE SPAs and 2 10GE SPAs. ...

Router Analysis, Inc. tested the Cisco ASR1004 and 1006 routers with the latest RP2, ESP40 and SIP40 hardware. The router was outfitted with Six 5 port GE SPAs and 2 10GE SPAs.

Overall the results were as expected, with the router performing within the specs provided by Cisco. For some tests the router performed at a lesser rate that expected.

Read the report to find out more.

Statistics

Views

Total Views
1,600
Views on SlideShare
802
Embed Views
798

Actions

Likes
0
Downloads
20
Comments
0

3 Embeds 798

http://www.routeranalysis.com 784
http://www.linkedin.com 13
http://moderation.local 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cisco ASR 1000 Series Testing Results and Analysis Cisco ASR 1000 Series Testing Results and Analysis Document Transcript

  • RA-070112-03 Cisco ASR 1000 Series Test Plan and Results July 2012 Temporary Text Cisco ASR1000 Series Router Test Plan and Results Report RA-070112-03 July 2012 1" Copyright © 2012 Router Analysis
  • RA-070112-03 Cisco ASR 1000 Series Test Plan and Results July 2012 Don’t Just Test. Analyze. " Quick Summary:Cisc o ASR1004 Router The ASR1004 with ESP40 and RP2 Works As ExpectedTest Plan and Re su lts When Configured With Features. ! Fabric Throughput Issues Do Not Impact QoS. REP OR T R A-0 70 11 2 -0 3 Router Supports NetFlow On A Few Interfaces Without Impacting Performance.Devices covered in this report: In this report Router Analysis tests the CiscoCisco: ASR1000 Series, specifically the ASR1004 along with theASR1004 RP2 / ESP40 ASR1006 in non-redundant mode.ARR1006 RP2 / ESP40 With the latest RP2 processor and ESP40 Fabric,How we test: how does the ASR1004 perform under stress?Utilizing state-of-the-art Router Analysis has created and executed a testtesting technology from plan to confirm Cisco’s claims about performance, featuresvendors such as IXIA and MU and QoS. We also check for head-of-line blocking issues. "Dynamics (now Spirent),Router Analysis has a world In general we had no issues with the ASR and itsclass testing lab. performance. The performance numbers mostly stayed above the ones quoted by Cisco. Read on to find out more. 2" Copyright © 2012 Router Analysis
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Table of Contents Table of Contents ................................................................................ 3! Results Summary ................................................................................. 4! Test Lab Setup .................................................................................... 5! Tests Carried Out ................................................................................. 5! Testing Results Reviewed .................................................................... 6! Appendix A – Test Cases ................................................................... 17! Appendix B – Test Result Data Tables ............................................... 21! About Router Analysis ........................................................................ 22! Table of Figures Figure 1 - Cisco ASR1004 IPv4 and IPv6 Forwarding ........................... 4 Figure 2 – Cisco ASR1004 ................................................................... 5 Figure 3 - Cisco ASR1004 IPv4 and IPv6 Forwarding ........................... 6 Figure 4 - ACL + uRPF Results ............................................................ 8 Figure 5 - At < 50% Per Port We Saw No Issues ................................. 9 Figure 6 - QoS Loss On Tos 0 at 57% Line Rate .................................. 9 Figure 7 - QoS Loss on TOS 0 at 57% Line Rate ............................... 10 Figure 8 - At 69% TOS 4 is affected ................................................... 11 Figure 9 - 39% Loss On TOS 4 .......................................................... 11 Figure 10 - 88.52% Caused issues With All Streams .......................... 12 Figure 11 - Background Traffic ........................................................... 12 Figure 12 - Background Traffic 10GE ................................................. 13 Figure 13 - QoS + BG Traffic.............................................................. 13 Figure 14 - QoS + BG Traffic - Highlighted ......................................... 13 Figure 15 - HOLB Testing .................................................................. 14 Figure 16 - OSPF Neighbors .............................................................. 15 Figure 17 - RSVP Tunnels .................................................................. 15 Figure 18 - MPLS TE Traffic ............................................................... 16 3 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Results Summary The Cisco ASR1004 performed within Cisco’s claimed specifications when tested at the Router Analysis test lab. Featureless IPv4 forwarding performed better than advertised: forwarding at 27 Million packets per second (Mpps) versus the claimed 23Mpps. Featureless IPv6 forwarding was tested at 21Mpps. Enabling features such as ACLs and uRPF dropped the routers forwarding performance down to 23.4Mpps for IPv4 and 11.7Mpps for IPv6. Both numbers are above the 10.4Mpps minimum forwarding rate of the system advertised by Cisco. QoS functioned as expected, protecting streams as configured even with fabric congestion. No head-of-line blocking was observed. 30000000 25000000 20000000 Claimed 15000000 Observed w uRPF 10000000 5000000 0 IPv4 Base IPv4 ACL IPv6 Base IPv6 ACL Figure 1 - Cisco ASR1004 IPv4 and IPv6 Forwarding ! 4 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Test Lab Setup The Router Analysis test lab utilizes the latest in IXIA testing hardware. The tests in this report were performed using an IXIA testing chassis with 22 Gigabit Ethernet ports, two 10 Gigabit Ethernet ports along with IXIAs IxNetwork, IxLoad and IxAutomate software. The ASR1004 was configured with one RP2, one ESP40 and two SIP40 carrier cards. Each carrier card had three five port Gigabit Ethernet v2 SPAs and one Single Port 10 Gigabit Ethernet v2 SPA. Figure 2 – Cisco ASR1004 Tests Carried Out • IP forwarding rate o Minimum Packet Size (64 bytes) ! With ACLs (permit any any) ! With uRPF ! With QoS ! With NetFlow o Packet IMIX + Features o 1500 Byte Packets (Fabric Throughput) + Features • Oversubscription o One to Many (HOLB Test) o Many to One o QoS • MPLS o RSVP-TE Scale and Performance 5 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Testing Results Reviewed IPv4 and IPv6 Forwarding Results with Features 30000000 25000000 20000000 Claimed 15000000 Observed w uRPF 10000000 5000000 0 IPv4 Base IPv4 ACL IPv6 Base IPv6 ACL Figure 3 - Cisco ASR1004 IPv4 and IPv6 Forwarding The main takeaway from the results is that the ASR1004 performed above the quoted specifications for featureless IPv4 based forwarding. IPv4 + uRPF also came in above the stated 23Mpps specification. IPv4 forwarding with ACLs was 21.3Mpps (about the same as IPv6 base which is 21Mpps). While testing the ASR1004, the lowest performance appeared while executing the “IPv6 forwarding with ACLs, uRPF, QoS and NetFlow” test case. By the time I had NetFlow configured on all of the interfaces the ASR1004 performance was down below 1Mpps. Most if not all companies would never apply NetFlow inbound and outbound on ALL interfaces. Running NetFlow on a few interfaces on the ASR1004 does not cause any issues. Both the IMIX and Fabric Bandwidth tests were above the minimum threshold and able to be done at full fabric bandwidth (40G). The Fabric numbers are uni-directional, if traffic is counted both directions we exceeded 80G across the system. 6 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! IPv4/IPv6 Forwarding + Features Tests Step 1: Configure 40G of ports to send traffic in a mesh Result: No Loss Seen Step 2: Configure a simple permit all ACL on the router: Step 3: Configure the ACL inbound on one interface Result: No Loss Seen Step 4: Configure the ACL outbound on one interface Result: No Loss Seen Step 4: Configure the ACL inbound and outbound on all interfaces Result: Traffic forwarding rate drops to 21Mpps Step 5: Configure traffic to run at about 21Mpps Step 6: Configure uRPF on one interface Result: No Loss Seen (Hard to get the traffic perfect) Step 7: Configure uRPF on a second interface Result: Drop in performance of 70Kpps Step 8: Configure uRPF on the rest of the interfaces checking the stats as you go along. Result: uRPF appears to cause a 25Kpps impact per GE port and a 250Kpps impact per 10GE port. 7 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! ! Figure 4 - ACL + uRPF Results QoS Testing and Results: From our tests we have determined that QoS works as expected on the system. In a simple test involving three GbE links (a two to one oversubscription). Three different TOS bits were set (0, 4 and 7) and traffic was sent balanced (33.3% per TOS) from each GbE towards a single GbE port outbound. We configured the router so that TOS 0 was protected up to 1% of line rate, TOS 4 was protected up to 40% of line rate and TOS 7 was protected up to 70% of line rate. When running the test we saw the following results: < 100% all streams went through without issue > 100% TOS 0 started to be dropped ~ 140% TOS 4 started to be dropped ~ 190% TOS 7 started to be dropped Step 1: Send less than 50% line-rate from each of the two ports 8 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Figure 5 - At < 50% Per Port We Saw No Issues Result: Up to almost 50% line-rate from the 2 ports (< 100% outbound on other interface) we saw no loss. Above 50% we saw loss for the TOS 0 stream but not the TOS 4 or 7 streams. Step 2: Send 57% line-rate from each the two ports (114% total) Figure 6 - QoS Loss On Tos 0 at 57% Line Rate 9 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Result: At about 57% we start to see TOS 0 traffic drop as shown in the magnified section below: Figure 7 - QoS Loss on TOS 0 at 57% Line Rate ! Step 3: Up the traffic to about 69% line-rate per port (138% total) 10 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Figure 8 - At 69% TOS 4 is affected As shown in this magnified section we see about 38% Loss on TOS 4 when sending 69% line-rate per port (138% line-rate outbound) Figure 9 - 39% Loss On TOS 4 11 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Step 4: Raise the traffic to about 88.5% we start seeing traffic loss on all streams. Figure 10 - 88.52% Caused issues With All Streams ! ! Result: We see loss on all 3 QoS traffic profiles. Final Result: From this basic test we have determined that QoS works as expected. ! QoS + Background Traffic + Fabric Loss/Forwarding Issues We then did the same QoS test with background traffic between all other ports on the system. Figure 11 - Background Traffic ! 12 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! The background traffic saw packet loss as seen above and magnified below: Figure 12 - Background Traffic 10GE Here you can see we are sending 14Mpps and only getting 4Mpps on the 10GE ports. And QoS traffic continued to function as expected: Figure 13 - QoS + BG Traffic Figure 14 - QoS + BG Traffic - Highlighted As you can see there are no drops on the TOS 4 or TOS 7 traffic. 13 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! ASR1004 Policy Configuration Here is the router configuration: Policy Map ixia-test2 Class prec_0 bandwidth remaining ratio 10 police cir percent 1 conform-action transmit exceed-action transmit violate-action drop Class prec_4 police cir percent 40 conform-action transmit exceed-action transmit violate-action drop bandwidth remaining ratio 10 Class prec_7 police cir percent 70 conform-action transmit exceed-action transmit bandwidth remaining ratio 10 Head-of-Line Blocking Testing: For HOLB Testing we setup a test case where we sent traffic from three GE ports to one GE port, then from another GE port to both the over-loaded GE port and an empty GE port. The red * indicates that the IXIA did not have real time information at the time I took the screen shot. Figure 15 - HOLB Testing ! ! ! 14 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! MPLS Testing and Results Step 1: Enable OSPF with MPLS Traffic Engineering Cisco Example: router ospf 1 mpls traffic-eng area 0 Step 2: Enable MPLS Traffic Engineering on the router and interfaces Cisco Example interface GigabitEthernet1/0/2 description to IXIA G3_2 ip address 198.18.8.1 255.255.255.0 ip ospf 1 area 0 mpls traffic-eng tunnels Step 3: Configure the IXIA to do OSPF with the interfaces on the router ! Figure 16 - OSPF Neighbors Step 4: Configure the IXIA to generate 100 fully meshed tunnels on each interface (1200 tunnels total). ! Figure 17 - RSVP Tunnels 15 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Step 5: Setup fully meshed traffic between all ports and run traffic to verify functionality. ! Figure 18 - MPLS TE Traffic Step 6: Increase the number of tunnels until you can no longer forward traffic, you see failures on the system or any functionality is compromised. Result: Tested up to 4800 TE Tunnels without issue. ! 16 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Appendix A – Test Cases The following two test cases represent a skeleton format of the set of tests performed in the Router Analysis lab. Test Case One – IP Forwarding Synopsis: When testing systems which have a set PPS or Fabric Bandwidth which is lower then the combined interfaces available; it is necessary to validate the best-case base forwarding rate. Once the rate has been determined, it is possible to judge other test cases against the best-case forwarding rate. Goal of Test: To confirm best-case IPv4 or IPv6 forwarding rate using the minimum and maximum packet size accepted by the system. Procedure: Configure the minimum number of links between the tester and the system under test (SUT) to provide either a.) More than the maximum PPS or b.) More than the maximum fabric bandwidth as claimed by the vendor. For example: The ASR1004 ESP40 claims a maximum of 23Mpps or 40Gbps. To test 23Mpps, you could use two 10GE interfaces running 64 byte packets at line rate (14.8Mpps x 2). To test the maximum fabric bandwidth you run large packets such as 1500 bytes and would need at least 42G of interfaces such as four 10GE and two GE (due to packet overhead which would be stripped before it reached the fabric). Data to Collect: Maximum PPS for IPv4, IPv6 and a 50/50 mix of IPv4 and IPv6 Maximum fabric bandwidth for IPv4, IPv6 and a 50/50 mix of IPv4 and IPv6 17 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Test Case Two – IP Forwarding with Features Synopsis: Once the best-case forwarding rate for IP has been determined, it is important to start enabling features to see what impact they may have on the forwarding rate. Goal of Test: To determine which features if any impact the forwarding rate of the SUT. Procedure: ACLs: Using your best-case setup from test one, apply a simple “allow all” access list to one interface inbound. Once you have recorded the data, swap the access list to outbound on the same interface. Next apply the access list both in and out on the same interface. The final steps are to apply the access list in and out on the other interfaces observing if the impact is the same or greater. When you have completed the testing you should have the access list applied to all interfaces inbound and outbound. (If no impact is seen even with all interfaces running ACLs, you may need to configure a bigger ACL using multiple Access List Entries (ACEs) with random source/destination addresses and ports blocked before an allow all statement.) If ACLs have no impact on forwarding performance it is possible that the SUT does all access list actions in hardware and no impact will be seen. Consult with your vendor to discuss. uRPF: For uRPF follow the same concept as with ACLs: apply uRPF to one interface at a time, log the results and then continue. 18 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! NetFlow: For NetFlow follow the same concept, a simple capture all NetFlow configuration applied inbound and outbound QoS: For QoS, build a simple policer and add it to each interface inbound/outbound and see if there is any impact. Data to Collect: Screen captures and if possible csv data for each of the tests you execute. ! 19 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Test Case Three – QoS Testing Synopsis: ! This is a simple QoS test based on information from the IXIA BlackBook. We have modified it slightly to utilize more than 2 ports. Goal of Test: Determine if QoS functions as expected by creating three different TOS valued streams, 0, 4 and 7. On the router configure a traffic policy to prefer 7 over 4 and 4 over 0. Procedure: Using IxNetwork create three streams with TOS levels of 0, 4 and 7 on two interfaces going towards a single interface. On the outbound interface configure a QoS policy to prefer TOS 7 over TOS 4 and TOS 4 over TOS 0. Run traffic and slowly raise it until you see packets drop for 0, then 4 and then 7. Additive Tests: Add in background traffic from your earlier test where you saw packet loss due to features. Confirm that the QoS policy is still working as expected. Data to Collect: Packet loss data for each stream at different traffic levels. Screen Captures of the IxNetwork screen during the tests. ! 20 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! Appendix B – Test Result Data Tables The following tables represent a sample of the data collected during testing in the Router Analysis Lab. IPv4 Forwarding Results Data Test Claimed Test Results Pass/Fail IPv4 PPS no ACL 23,000,000 26,600,000 Pass IPv4 PPS with ACL 10,400,000 21,774,000 Pass IPv4 Fabric BW 40G 40G Pass IPv4 Fabric BW ACL 40G 40G Pass IPv4 + uRPF Forwarding Results Data Test Claimed Test Results Pass/Fail IPv4 + uRPF 10,400,000 23,450,000 Pass IPv4 ACL + uRPF 10,400,000 18,140,800 Pass IPv4 Fabric + uRPF 40G 40G Pass IPv4 Fabric ACL + uRPF 40G 40G Pass IPv6 Forwarding Results Data Test Claimed Test Result Pass/Fail IPv6 PPS no ACL 10,400,000 21,000,000 Pass IPv6 PPS with ACL 10,400,000 14,900,000 Pass IPv6 Fabric BW 40G 40G Pass IPv6 Fabric BW with ACL 40G 40G Pass IPv6 + uRPF Forwarding Results Data Test Claimed Test Result Pass/Fail IPv6 + uRPF 10,400,000 17,500,000 Pass IPv6 ACL + uRPF 10,400,000 14,900,000 Pass IPv6 Fabric + uRPF 40G 40G Pass IPv6 Fabric ACL + uRPF 40G 40G Pass ! 21 Copyright © 2012 Router Analysis !
  • RA-070112-03 Cisco ASR1000 Series Test Plan and Results July 2012! About Router Analysis With a world-class fully vendor independent testing lab, Router Analysis provides unbiased information to its clients. Router Analysis provides the following services: Router and Switch Testing Test Plan Creation On-Site Testing Support Router, Switch and Tester Training Network Design Consultation, Verification and Installation … And More. Contact Router Analysis by phone at +1 888 SONN COM (766-6266) or by email at info@routeranalysis.com for a quote, more information or any questions. 22 Copyright © 2012 Router Analysis !