This document discusses network virtualization and summarizes key concepts around network performance testing. It notes that current performance testing does not adequately model the diverse real-world network conditions users experience. Specifically, it states that testing is typically done from high-speed lab networks, whereas users connect via various networks including 3G, LTE, WiFi and broadband with differing bandwidth and latency characteristics. The document advocates for using network virtualization technology to simulate diverse user environments and more comprehensively test the impact of network conditions on performance and user experience.

1. Shunra University
Session 1
Introduction to Network
Virtualization
© Copyright 2013 Shunra Software, LTD. Information contained herein is subject to change without notice.

2. Let’s Begin with a Bit of
Theory
© Copyright 2013 Shunra Software, LTD. Information contained herein is subject to change without notice.

3. The Network Impact
New York
London
Tokyo
New York
Data Center
WAN
–
–
–
www.shunra.com
3
Network Virtualization for Software Testing

4. Network Impairments
www.shunra.com
4
Network Virtualization for Software Testing

5. The Mobile Network Impact
–
–
–
www.shunra.com
5
Network Virtualization for Software Testing

6. Mobile Network Impairments
www.shunra.com
6
Network Virtualization for Software Testing

7. Why is There Delay on a
Terrestrial Network?
A
t (msec) = d / (2/3)c
B
Signal
d
Lost Packet
Packet Packet Packet
Packet
BUSY
www.shunra.com
7
Network Virtualization for Software Testing

8. Signal Speed – MPLS WAN
New
York
London
Tokyo
Sydney
5,585km
New York
Frankfur
t
6,219km
10,871k
m
15,998k
m
639km
9,585km
16,991k
m
London
New York
London
Frankfurt
London
Frankfur
t
Tokyo
16,480k
m
Sydney
7,793km
28ms
31ms
54ms
80ms
3ms
48ms
85ms
47ms
82ms
Tokyo
Frankfurt
Tokyo
9,357km
New
York
New York
London
Frankfurt
Sydney
Tokyo
39ms
T = D / (2/3)c
www.shunra.com
8
Network Virtualization for Software Testing

9. Internet Consumer Latency
Broadband Latency (FCC)
“Fiber-to-the-home services provided 17 ms round-trip
latency on average, while cable-based services averaged
28 ms, and DSL-based services averaged 44 ms.”
Connection
Type
RTT
Fiber
17ms
Cable
28ms
DSL
44ms
http://www.fcc.gov/measuring-broadband-america
www.shunra.com
9
Network Virtualization for Software Testing

10. Mobile Latency – Last Mile + Backhaul
For more on this topic
checkout:
Ilya Grigorick
Mobile Performance from the
Radio Up session at Google IO
www.shunra.com
10
Network Virtualization for Software Testing

11. Mobile Latency – 1st. Data point
Ofcom study, UK Mobile Networks, May ‘11
“Latency… was an average of 117 milliseconds. Nearly two
Static Tests: thirds of tests took more than 100 milliseconds”
User Panel: “Average latency was 192 milliseconds and around a third
of tests delivered latency of higher than 200 milliseconds”
http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf
www.shunra.com
11
Network Virtualization for Software Testing

12. Mobile Latency – 2nd. Data point
Cisco Cloud Index, 2010-2015
“Global average fixed latency is 125 and average mobile
latency is 290… Global mobile consumer average latency is
307.3 ms”
www.shunra.com
12
Network Virtualization for Software Testing

13. Mobile Latency – Summary of Averages
Latency by Connec on Type
350
300
250
200
150
100
50
0
Fiber
www.shunra.com
Cable
DSL
Mobile
(Ofcom Tests)
13
Mobile
(Ofcom
Panel)
Mobile
(Cisco)
Mobile
Consumer
(Cisco)
Network Virtualization for Software Testing

14. –
Random Early Detection
–
Average queue monitored size & packets
dropped based on statistical probabilities. If
buffer is almost empty, all incoming packets are
accepted. As queue grows, probability of
dropping incoming packet grows. When buffer is
full, probability reaches 1 and all incoming
packets are dropped.
–
–
–
–
www.shunra.com
14
Network Virtualization for Software Testing

15. Packet Loss, Akamai/MIT Research
Tested
149 Million connections in 51 countries
– Compared PL for mobile networks from non-mobile networks
Most
connections have no packet loss (Median PL is 0%).
– 75th percentile usually zero too
Average
connection packet loss: 2.5%
– Fixed network average is 2.2%
Implies
connections are usually fine, but sometimes terrible
– The average PL for the non-zero half is 5%
Suggests
“small” connections
have higher Packet Loss
– HTTP Connections are often small
http://pam2012.ftw.at/papers/PAM2012paper6.pdf
www.shunra.com
15
Network Virtualization for Software Testing

16. 

No Jitter
Jitter
Jitter-Sensitive Applications:
• Video
• VoIP
www.shunra.com
16
Network Virtualization for Software Testing

17. 


–
WAN Capacity
(examples)
LAN Capacity
T3 - 45 Mbps
T1 – 1.5 Mbps
Mobile Capacity
(examples)
100 Mbps or
1 Gbps
128Kbps - 5 Mbps
www.shunra.com
17
Network Virtualization for Software Testing

18. Protocol
GPRS
CDMA200 (1X)
Edge
3G Bar
EV-DO
EDGE Evolution
HSDPA
HSPA+
4G Bar
HSPA+ Release 11
LTE
WiMax
www.shunra.com
Max Downlink
56-115 Kbps
153.6 Kbps
236.8 Kbps
2 Mbps stationary client
384 Kbps moving client
2.5-4.9 Mbps
1.6 Mbps
2 Mbps
14.4 Mbps
1 Gbps stationary client
100 Mbps moving client
21-672 Mbps
100-300 Mbps
37-365 Mbps
18
Max Uplink
0.15-1.8 Mbps
0.5 Mbps
200 Kbps
5.8-168 Mbps
50-75 Mbps
17-376 Mbps
Network Virtualization for Software Testing

19. You only get those speeds it if you’re this
guy…
www.shunra.com
19
Network Virtualization for Software Testing

20. PCWorld/Novarum
3G
–Download: 0.59 – 3.84 Mbps
–Upload: 0.56 – 1.44 Mbps
4G
–Download: 2.81 – 9.12 Mbps
–Upload: 0.97 – 5.86 Mbps
Date:
Apr, 2012
Source (PCWorld): http://bit.ly/JX7MQw
www.shunra.com
20
Network Virtualization for Software Testing

21. Test Time!
How
much impact does the difference in
Bandwidth & Latency have on web
performance?
Tested
Sites: Alexa US Top 100
Details:
–Increase Latency, Download & Upload Bandwidth
separately
–Set no limits on other two variables
–Run each test 3 times, use median result
www.shunra.com
Network Virtualization for Software Testing

22. Download BW Impact on Load
Time
Poor Cell Conn
DSL
Cable
FIBER
Avg Cell Conn
www.shunra.com
Network Virtualization for Software Testing

23. Upload BW Impact on Load Time
Poor Cell Conn
Avg Cell Conn
DSL
Cable FIBER
www.shunra.com
Network Virtualization for Software Testing

24. Latency Impact on Load Time
Poor Cell Conn
Avg Cell Conn
DSL
Cable
FIBER
www.shunra.com
Network Virtualization for Software Testing

25. Bandwidth/Latency Difference –
Conclusions for Web Performance
Upload
difference doesn’t matter much
until <200 KBPS
Download difference DOES matter
Latency difference matters the most
www.shunra.com
Network Virtualization for Software Testing

26. The Impact of Performance Optimization
–
–
–
–
www.shunra.com
26
Network Virtualization for Software Testing

27. Knowing all that…
How are Organizations
Testing Today?
© Copyright 2013 Shunra Software, LTD. Information contained herein is subject to change without notice.

28. Current Performance Testing Vs. Production
Data Center
Network
Site Type
Devices
Firewall
Automated
Testing
Manual
Testing
Branch Office
WAN
VPN
DSL/Cable
WiFi
4G 3G 2.5G
Etc.
Residential
Mobile Salesforce
Load
Testing
Road Warrior
Mobile (Anywhere)
www.shunra.com
28
Network Virtualization for Software Testing

29. MyAT&T Mobile App – Prod Vs. Test
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
AT&T Data Center
MyAT&T iPhone Application - Common User
Actions
14
12
10
8
6
4
2
0
WiFi Based Lab
WiFi based mobile tester
www.shunra.com
myAT&T – Dev/Test
11.6
9.2
6.8
5.3
1.6
4.4
4.1
1.3
Wifi

30. MyAT&T Mobile App – Prod Vs. Test
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
ATT Data Center
HP - Virtual Users
Hi Speed Lab Network
myAT&T – Dev/Test
www.shunra.com

31. Mobile End User Experience Testing with Shunra
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
ATT Data Center
Shunra Mobile Virtualization
Shunra mobile tester
www.shunra.com
myATT – Dev/Test

32. Mobile End User Experience Testing with Shunra
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
ATT Data Center
Shunra Mobile Virtualization
Mobile Labs Integration
www.shunra.com
myATT – Dev/Test