7. Why is There Delay on a
Terrestrial Network?
A
t (msec) = d / (2/3)c
B
Signal
d
Lost Packet
Packet Packet Packet
Packet
BUSY
www.shunra.com
7
Network Virtualization for Software Testing
8. Signal Speed – MPLS WAN
New
York
London
Tokyo
Sydney
5,585km
New York
Frankfur
t
6,219km
10,871k
m
15,998k
m
639km
9,585km
16,991k
m
London
New York
London
Frankfurt
London
Frankfur
t
Tokyo
16,480k
m
Sydney
7,793km
28ms
31ms
54ms
80ms
3ms
48ms
85ms
47ms
82ms
Tokyo
Frankfurt
Tokyo
9,357km
New
York
New York
London
Frankfurt
Sydney
Tokyo
39ms
T = D / (2/3)c
www.shunra.com
8
Network Virtualization for Software Testing
9. Internet Consumer Latency
Broadband Latency (FCC)
“Fiber-to-the-home services provided 17 ms round-trip
latency on average, while cable-based services averaged
28 ms, and DSL-based services averaged 44 ms.”
Connection
Type
RTT
Fiber
17ms
Cable
28ms
DSL
44ms
http://www.fcc.gov/measuring-broadband-america
www.shunra.com
9
Network Virtualization for Software Testing
10. Mobile Latency – Last Mile + Backhaul
For more on this topic
checkout:
Ilya Grigorick
Mobile Performance from the
Radio Up session at Google IO
www.shunra.com
10
Network Virtualization for Software Testing
11. Mobile Latency – 1st. Data point
Ofcom study, UK Mobile Networks, May ‘11
“Latency… was an average of 117 milliseconds. Nearly two
Static Tests: thirds of tests took more than 100 milliseconds”
User Panel: “Average latency was 192 milliseconds and around a third
of tests delivered latency of higher than 200 milliseconds”
http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf
www.shunra.com
11
Network Virtualization for Software Testing
12. Mobile Latency – 2nd. Data point
Cisco Cloud Index, 2010-2015
“Global average fixed latency is 125 and average mobile
latency is 290… Global mobile consumer average latency is
307.3 ms”
www.shunra.com
12
Network Virtualization for Software Testing
13. Mobile Latency – Summary of Averages
Latency by Connec on Type
350
300
250
200
150
100
50
0
Fiber
www.shunra.com
Cable
DSL
Mobile
(Ofcom Tests)
13
Mobile
(Ofcom
Panel)
Mobile
(Cisco)
Mobile
Consumer
(Cisco)
Network Virtualization for Software Testing
14. –
Random Early Detection
–
Average queue monitored size & packets
dropped based on statistical probabilities. If
buffer is almost empty, all incoming packets are
accepted. As queue grows, probability of
dropping incoming packet grows. When buffer is
full, probability reaches 1 and all incoming
packets are dropped.
–
–
–
–
www.shunra.com
14
Network Virtualization for Software Testing
15. Packet Loss, Akamai/MIT Research
Tested
149 Million connections in 51 countries
– Compared PL for mobile networks from non-mobile networks
Most
connections have no packet loss (Median PL is 0%).
– 75th percentile usually zero too
Average
connection packet loss: 2.5%
– Fixed network average is 2.2%
Implies
connections are usually fine, but sometimes terrible
– The average PL for the non-zero half is 5%
Suggests
“small” connections
have higher Packet Loss
– HTTP Connections are often small
http://pam2012.ftw.at/papers/PAM2012paper6.pdf
www.shunra.com
15
Network Virtualization for Software Testing
21. Test Time!
How
much impact does the difference in
Bandwidth & Latency have on web
performance?
Tested
Sites: Alexa US Top 100
Details:
–Increase Latency, Download & Upload Bandwidth
separately
–Set no limits on other two variables
–Run each test 3 times, use median result
www.shunra.com
Network Virtualization for Software Testing
22. Download BW Impact on Load
Time
Poor Cell Conn
DSL
Cable
FIBER
Avg Cell Conn
www.shunra.com
Network Virtualization for Software Testing
23. Upload BW Impact on Load Time
Poor Cell Conn
Avg Cell Conn
DSL
Cable FIBER
www.shunra.com
Network Virtualization for Software Testing
24. Latency Impact on Load Time
Poor Cell Conn
Avg Cell Conn
DSL
Cable
FIBER
www.shunra.com
Network Virtualization for Software Testing
25. Bandwidth/Latency Difference –
Conclusions for Web Performance
Upload
difference doesn’t matter much
until <200 KBPS
Download difference DOES matter
Latency difference matters the most
www.shunra.com
Network Virtualization for Software Testing
26. The Impact of Performance Optimization
–
–
–
–
www.shunra.com
26
Network Virtualization for Software Testing
28. Current Performance Testing Vs. Production
Data Center
Network
Site Type
Devices
Firewall
Automated
Testing
Manual
Testing
Branch Office
WAN
VPN
DSL/Cable
WiFi
4G 3G 2.5G
Etc.
Residential
Mobile Salesforce
Load
Testing
Road Warrior
Mobile (Anywhere)
www.shunra.com
28
Network Virtualization for Software Testing
29. MyAT&T Mobile App – Prod Vs. Test
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
AT&T Data Center
MyAT&T iPhone Application - Common User
Actions
14
12
10
8
6
4
2
0
WiFi Based Lab
WiFi based mobile tester
www.shunra.com
myAT&T – Dev/Test
11.6
9.2
6.8
5.3
1.6
4.4
4.1
1.3
Wifi
30. MyAT&T Mobile App – Prod Vs. Test
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
ATT Data Center
HP - Virtual Users
Hi Speed Lab Network
myAT&T – Dev/Test
www.shunra.com
31. Mobile End User Experience Testing with Shunra
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
ATT Data Center
Shunra Mobile Virtualization
Shunra mobile tester
www.shunra.com
myATT – Dev/Test
32. Mobile End User Experience Testing with Shunra
Mobile Customers
3G
LTE
Edge
WiFi
AT&T Cloud Partner Services
ATT Data Center
Shunra Mobile Virtualization
Mobile Labs Integration
www.shunra.com
myATT – Dev/Test
Editor's Notes
You can’t deploy a load generator on a mobile network