Comparing IPv6 and IPv4 Performance, by John Berg. A presentation given at APNIC 38 during the IPv6 Plenary session.

1. Comparing IPv6 and IPv4 Performance
John Berg – Lead Engineer September 16, 2014
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 1

2. Background
• CableLabs has been conducting IPv6 interops since 2009
– Observed subtle but persistent IPv6 performance benefit
• Collected real performance data from one of our member operators
• Conducted lab testing to measure IPv4/IPv6 performance
– Native IPv6
– IPv4 with one layer of NAT
– IPv4 with two layers of NAT
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 2
2

3. How Service Provider Data Was Collected
• Service Provider produced >1.5 million IPv4/IPv6 data records
– Analysis performed jointly by CableLabs and MSO
• Metrics collected on data records:
– Average Round Trip Time (AVGRTT) for a variety of popular IPv4 and IPv6 web sites
– Statistics Include: Minimum, Maximum, Mean, Median, Standard Deviation, etc.
• Measurement agents configured on backbone POPs in ten hubs
– Randomized latency checks executed each hour
– RTT interval = completion of TCP handshake between client and server
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 3
3
TCP SYN
SYN + ACK
TCP ACK

4. Where the Data Was Collected
San Jose
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 4
4
New York
Washington, DC
Charlotte
Atlanta
Houston
Dallas/Ft.
Worth
Los Angeles
Chicago
Seattle

5. Is IPv6 Really Faster?
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 5
5
These visits to apple.com make it appear that is so…..

6. Is IPv6 Really Faster?
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 6
6
RTT for wikipedia.org is still measurably faster for IPv6 on average…..

7. Is IPv6 Really Faster?
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 7
7
But on Facebook RTT for IPv6 and IPv4 are nearly identical…..

8. Is IPv6 Really Faster?
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 8
8
Again, nearly identical results for Netflix…..

9. Is IPv6 Really Faster?
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 9
9
And this time IPv4 is significantly faster, by nearly 25 ms…..

10. Is IPv6 Really Faster?
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 10
10
IPv4 wins again. What are we to make of these results?

11. IPv4 vs IPv6 Aggregate View
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 11
11

12. Lab Testing
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 12
12

13. Lab Testing Methodology
• Conducted matched IPv4-IPv6 ping tests
over the course of an IPv6 interop
– First sets: 2 NATS enabled
– Second sets: 1 NAT enabled
• Identify effect of NAT without IPv4-IPv6
path differences
• Did NOT measure effects of traffic
engineering or differentiated peering
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 13
13

14. Results – One Layer of NAT
• Could NAT be responsible for slower IPv4
performance?
• IPv6 performed measurably better than
IPv4
– Lower mean, median, standard deviation, and
minimum
– Statistically significant differences
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 14
14
Common Home Network Scenario

15. Results – Two Layers of NAT
• What happened?
• One NAT showed differences, two NATs
show nearly identical results
– 81% chance that the means are really the same*
– IPv4 in both cases still shows higher variability
– IPv6 performed better half the time across 6 test
runs
• Could NAT performance vary per device?
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 15
15
Possible CGN Effect
*Based on Students t-test analysis

16. Interpreting The Results Data
• How do we explain the disparity in test results?
– Analysis shows a subtle IPv6 performance advantage
– However, there are more questions than answers….
• Some possible explanations:
– Does NAT performance vary across some IPv4 platforms?
– Is hop count a factor for IPv4 vs. IPv6?
– How do network effects, such as tunnels, traffic engineering and congestion factor in?
– What about Content Provider IPv6 transition strategies?
• Better tools and methodologies for gathering end-to-end performance
metrics are needed
• Bottom line – further research is needed
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 16
16

17. CableLabs Initiative
• CableLabs is developing performance measurement tools and
methodologies as part of our IP Performance Evaluation & Reporting
(CLIPPER) project
• Project will standardize testing methodologies for MSOs
– Aligns with IETF IPPM & LMAP and BBF WT-143 working groups
– Introduce common test platform
– Gather data on jitter, delay, packet loss, throughput and DNS response time for both
IPv4 and IPv6
– Better correlation of speed tests between home router and wireless devices
• Recommendation: Systematic performance metric collection should
consider differences between IPv6 and IPv4
© Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable. 17
17

18. © Cable Television Laboratories, Inc. 2014. Do not share this material with anyone other than CableLabs Members, and vendors under CableLabs NDA if applicable.