A preliminary test for broadband speed measurement

  • 272 views
Uploaded on

 

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
272
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A preliminary test for broadband speed measurement Toshiya Jitsuzumi Kyushu University Jitsuzumi@econ.kyushu-u.ac.jp Jan. 19, 2014
  • 2. Question •How large is the variation in speed measurement results and how does it affect the measurement? •Is it necessary to measure things other than download speed?
  • 3. Part 1: Fixed (& Wi-Fi) broadband
  • 4. Test Setting • Connected to the speedtest.net website and measured the download/upload speeds and latency as frequently as possible during 30 minutes. • Date and time Jan. 18, 2014 14:00-14:30 (Japan Time) • Measurement place and Internet setting My office at Kyushu University equipped with the university’s wired/wireless LAN. Router: LYNKSIS Wireless-N Home Router (WRT150N)
  • 5. Measurement site • Speedtest server at Tokyo, hosted by Alocac, Inc. • Network path from my office [133.5.240.168] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 203 ms 1 ms 1 ms 2 ms 1 ms 13 ms 13 ms 21 ms 52 ms 22 ms 23 ms 23 ms * * * 23 ms 1 ms 1 ms 1 ms 2 ms 1 ms 13 ms 13 ms 17 ms 15 ms 22 ms 22 ms 22 ms * * * 22 ms 1 ms 1 ms 2 ms 2 ms 1 ms 13 ms 13 ms 17 ms 15 ms 47 ms 23 ms 22 ms * * * 23 ms 192.168.1.1 router.en.kyushu-u.ac.jp [133.5.240.254] 133.5.69.241 133.5.9.230 150.99.191.41 osaka-dc-rm-ae2-vlan10.s4.sinet.ad.jp [150.99.2.41] osaka-dc-gm1-ae0-vlan10.s4.sinet.ad.jp [150.99.2.58] 103.246.232.94 obpjbb206.int-gw.kddi.ne.jp [118.155.199.73] otejbb206.int-gw.kddi.ne.jp [59.128.4.181] cm-ote232.int-gw.kddi.ne.jp [118.155.197.13] 111.87.16.42 Request timed out. Request timed out. Request timed out. kvps-27-34-160-198.secure.ne.jp [27.34.160.198]
  • 6. Test parameters • Three PCs were used for speed measurement. • DELL XPS 8500 [Wireline 1] • Windows 7 Ultimate Service Pack1, Intel® Core™ i7-3770 CPU @3.40GHz • RAM 16.0GB, 64bit Operating System • Connected to the university’s LAN via cable • DELL XPS 420 [Wireline 2] • Windows Vista Home Premium Service Pack2, Intel® Core™ 2 QuadCPU Q9550 @2.83GHz • RAM 4.0GB, 32bit Operating System • Connected to the university’s LAN via cable • MacBook Air [Wi-Fi] • OS X version 10.9.1, 1.6GHz Intel Core 2 Duo • RAM 4GB 1067MHz DDR3 • Connected to the university’s LAN via Wi-Fi
  • 7. Result 1: DL SPD Wireline 1 59 61 60 Average 38.1Mbps 36.0 17.9 Standard dev. 8.0Mbps 8.4 2.1 Coefficient of variation 70 Wi-Fi N Download Speed (Mbps)(2013/01/18 14:00-14:30) Wireline 2 0.209 0.233 0.116 60 50 40 30 20 10 0 0 5 10 15 Wireline 1 Wireline 2 20 Wi-Fi 25 30
  • 8. Result 2: UL SPD Wireline 1 59 61 60 Average 34.8Mbps 24.8 16.9 Standard dev. 9.1Mbps 7.0 0.8 Coefficient of variation 60 Wi-Fi N Upload Speed (Mbps)(2013/01/18 .14:00-14:30) Wireline 2 0.262 0.280 0.048 50 40 30 20 10 0 0 5 10 15 Wireline 1 Wireline 2 20 Wi-Fi 25 30
  • 9. Result 3: Latency Wireless 1 59 61 60 Average 25.6ms 16.6 31.4 Standard dev. 1.78ms 3.73 2.13 Coefficient of variation 45 Wi-Fi N Latency (ms) (2013/01/18 14:00-14:30) Wireless 2 0.070 0.224 0.068 40 35 30 25 20 15 10 5 0 0 5 10 15 Wireline 1 Wireline 2 20 Wi-Fi 25 30
  • 10. Result 4: DL vs. UL Download vs. Upload 60 Upload Speed (Mbps) 50 UL = 33.3 + 0.04 DL − 9.91D 𝑤2 − 17.1D 𝑤𝑖𝑓𝑖 40 𝑝= 0.58 30 Note: UL=upload speed (Mbps); DL=download speed (Mbps); Dw2 = 1 if the data comes from Wireline 2, or = 0, otherwise; Dwifi = 1 if the 20 data comes from Wi-Fi, or = 0, otherwise; and p is p-value. 10 0 0 10 20 30 40 50 Download Speed (Mbps) Wireline 1 Wireline 2 Wi-Fi 60 70
  • 11. Result 5: DL vs. Latency Download vs. Latency 40 Latenvy (ms) 35 LAT = 27.4 − 0.05 DL − 9.06D 𝑤2 + 4.84D 𝑤𝑖𝑓𝑖 30 𝑝= 0.10 25 Note: LAT=latency (ms); DL=download speed (Mbps); Dw2 = 1 if the data comes from Wireline 2, or =0, otherwise; Dwifi = 1 if the data 20 comes from Wi-Fi, or =0, otherwise; and p is p-value. 15 10 0 10 20 30 40 50 Download Speed (Mbps) Wireline 1 Wireline 2 Wi-Fi 60 70
  • 12. Preliminary findings and implications • Variation in broadband speed measurement was quite large and therefore it is important to gather a large number of measurements to obtain robust results. • Surprisingly, Wi-Fi connection generated more stable results than wired connection. • Latency measurement had less variation than speed measurement. • In measuring QoS of fixed broadband, Wi-Fi or wired must be clearly addressed. • Wi-Fi connection downgraded the speed by half, and the latency by 20 to 90%. • Measuring download speed only may not be sufficient to describe the overall QoS. • Correlation between download speed and the other measures was not significant.
  • 13. Part 2: Mobile broadband
  • 14. Test Setting • Install the iPhone App “OOKLA SPEEDTEST” and measured the download/upload speeds and latency as frequently as possible during 30 minutes. • Date and time Jan. 19, 2014 14:00-14:30 (Japan Time) • Measurement setting iPhone 5 and iPad 2 on my desk at Kyushu University.
  • 15. Test parameters and measurement site • Two mobile terminals were used for speed measurement. • iPhone 5 [LTE] • iOS 7.0.4 • 16GB model • Disabled the Wi-Fi function and connected to the LTE network of Softbank • iPad 2 [3G] • iOS 7.0.4 • 64GB model • Disabled the Wi-Fi function and connected to the 3G network of Softbank • Speedtest server at Tokyo, hosted by Alocac, Inc.
  • 16. Path to the site 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 1.479ms 48.269ms 53.922ms 63.959ms 78.297ms 81.238ms 76.054ms * 102.022ms * 113.069ms 79.172ms 54.858ms * 112.238ms 81.309ms 60.359ms 78.349ms 81.181ms 81.927ms 79.257ms 91.198ms 77.631ms 78.086ms 79.301ms 96.986ms 79.105ms 56.267ms * * * 116.654ms 0.772ms 50.246ms 40.201ms 58.92ms 53.911ms 79.041ms 57.761ms * 61.808ms * 60.569ms 60.716ms 78.124ms * 62.091ms 63.087ms 61.717ms 79.309ms 62.049ms 64.81ms 61.105ms 57.532ms 62.634ms 59.822ms 47.306ms 99.193ms 98.464ms 95.245ms * * * 88.744ms 0.793ms 57.111ms 54.193ms 81.512ms 78.864ms 77.41ms 96.612ms * 80.319ms * 80.519ms 64.358ms 78.042ms * 78.308ms 95.298ms 80.3ms 61.638ms 79.951ms 80.356ms 74.447ms 78.262ms 81.464ms 79.421ms 79.344ms 79.359ms 80.246ms 78.588ms * * * 80.311ms 172.20.10.1 192.168.120.65 192.168.120.66 192.168.120.116 192.168.120.113 192.168.120.113 192.168.120.124 * pw126211191058.5.panda-world.ne.jp * 172.28.62.12 172.28.62.27 172.28.62.27 * pw126211191113.5.panda-world.ne.jp pw126211186130.5.panda-world.ne.jp pw126211184097.5.panda-world.ne.jp pw126211184041.5.panda-world.ne.jp 101.110.21.121 10.7.196.21 10.7.61.21 10.2.194.125 10.9.194.142 210.132.127.165 obpjbb206.int-gw.kddi.ne.jp otejbb206.int-gw.kddi.ne.jp cm-ote232.int-gw.kddi.ne.jp 111.87.16.42 * * * kvps-27-34-160-198.secure.ne.jp
  • 17. Result 6: DL SPD LTE N 20 15 10 5 0 0 5 10 15 DL_3G 20 DL_LTE 25 30 13.46Mbps 6.08 4.88Mbps 1.04 Coefficient of variation 25 18 Standard dev. 30 63 Average Download speed (Mbps) (2013/01/19 14:00-14:30) 3G 0.362 0.171
  • 18. Result 7: UL SPD LTE N 12 10 8 6 4 2 0 0 5 10 15 UL_3G 20 UL_LTE 25 30 7.55Mbps 0.75 2.96Mbps 0.34 Coefficient of variation 14 17 Standard dev. 16 63 Average Upload speed (Mbps) (2013/01/19 14:00-14:30) 3G 0.392 0.456
  • 19. Result 8: Latency LTE N 80 60 40 20 0 0 5 10 Latency_3G 15 20 Latency_LTE 25 30 57.67ms 79.22 6.85ms 9.30 Coefficient of variation 100 18 Standard dev. 120 63 Average Latency (ms) (2013/01/19 14:00-14:30) 3G 0.119 0.117
  • 20. Result 9: DL vs. UL Download vs. Upload Upload Speed (Mbps) 16 14 12 UL = −0.88 + 0.27 DL + 4.83D 𝐿𝑇𝐸 10 𝑝<0.01 8 6 Note: UL=upload speed (Mbps); DL=download speed (Mbps); DLTE = 1 if the data comes from LTE or = 0, otherwise; and p is p-value. 4 2 0 0 5 10 15 20 Download Speed (Mbps) UL_3G UL_LTE 25 30
  • 21. Result 10: DL vs. Latency Download vs. Latency 120 Latency (ms) 100 LAT = 82.4 − 0.46 DL − 18.5D 𝐿𝑇𝐸 80 𝑝= 0.02 60 Note: LAT=latency (ms); DL=download speed (Mbps); DLTE = 1 if the 40 data comes from LTE or = 0, otherwise; and p is p-value. 20 0 0 5 10 15 20 Download Speed (Mbps) Latency_3G Latency_LTE 25 30
  • 22. Preliminary findings and implications • Variation in LTE broadband measurement was larger than that of fixed broadband in terms of download/upload speed, and therefore it needs a larger number of samples. • Because the path to the measurement site was much longer in the mobile broadband, there seems to exist more factors that affected the E2E performance than in the fixed counterpart. • Compared to fixed broadband, download speed figures have more representative values in mobile broadband. • Correlation between download speed and the other measures was highly significant.
  • 23. Agenda for future research I need three things to move forward: 1. more data, 2. more data, and 3. more data.