2. 1
Final Project
Table of Contents
Executive Summary
Current State of the Industry 2
Background and Summary Of Other Findings/Problem Statement
Qualcomm(most frequently cited proponent of LTE-U): 3
Introduction to Our Project
Test Objectives/ Goals
Items Used 5
Test Methodology 6
Client-Server Model 6
2.4 GHz Pretest Findings
5 GHz Main Testing and Findings
Simultaneous Downstream/Upstream Test Findings
Our Results and Analysis………………………………………………………………………………………………….. 14
Conclusion
Solution/Recommendations 6
Works Cited 7
3. 2
Executive Summary
The goal of our project of multi-vendor channel sharing was to test and find out whether
or not different makes/vendors of Wi-Fi access points share unequal bandwidth when they are
operating on the same channel. We wanted to see if the connection properties differ with
different vendor access point vs. same vendor access point when sharing channels. We tested
upstream and downstream in 4 different clockwise positions on both 2.4 GHz and 5 GHz to be
able to see if there were any trends. Ultimately, we wanted to find out if different chipsets
between two access points on the same channel had any adverse effect the way bandwidth is
propagated, whether or 2.4 GHz or 5 GHz.
We first started comparing same vendor access point to find how they act in the same
environment/channel and later we tested two different access point makes with different chipsets.
The point of this testing and analysis was to find whether or not unequal bandwidth sharing
occurs between different vendor access points on the same channel, however you will later see
our finding on how our hypothesis was true, access points sharing on the same channel with
same vendor is more equal than sharing with different vendor access points.
Current State of the Industry
As it stands in the current state of the Mobile Information Industry, there is trend in the
unlicensed frequencies that cover the WIFI spectrum that LTE-U companies want to use to offer
their service. Wi-Fi providers do not want LTE-U to use these frequencies to operate. In fact,
many stakeholders in the industry such as Google, Broadcom and Wi-Fi Alliance have begun to
weigh in on the newly Federal Communication Commission’s (FCC) request for information on
current trends related to LTE-Unlicensed (LTE-U) and Licensed Assisted Access (LAA).
These stakeholders in the Wi-Fi community have made it clear that the LTE-U/LAA and
Wi-Fi communities need a better collaboration in order to have the sharing of unlicensed
spectrum worked out. Those that want to implement LTE-U and LAA, surprisingly argue that
many collaboration has occurred since Verizon and their technology partners formed from the
LTE-Unlicensed Forum established in April of 2015.
Since the FCC has called on the industry to supply comments on LTE/LAA after a
number of organizations are concerned about LTE-U/LAA collaboration which could potentially
interfere with or have a tremendous effect on users or future users of the unlicensed spectrum.
Lastly, essentially what the Wi-Fi community is concerned about is that LTE-U could possibly
bully Wi-Fi users out of their wireless space. And LTE-U/LAA argues that their existence will
make Wi-Fi collaborate better with stakeholders in the Wi-Fi community.
4. 3
Background and Summary of Other Findings/Problem Statement
Here is another finding that corresponds to use and controversy of LTE over unlicensed
spectrum and also our project concerning multi-vendor channel sharing, as are they both
subjected the same testing for equal channel sharing frequencies between 4G LTE radios and
Wi-Fi devices in the GHz band. Because the use of LTE over unlicensed spectrum is heading
toward field/trials within recent years. These testing have shown mixed results as to whether
LTE-U or Wi-Fi can exist side-by-side fairly in Wi-Fi community without either of the two
overtaking another. Since the concept of what is considered to be “fair” can only be seen as fair
in one's own eye.
The finding shows the studies have not been done an equal footing, both LTE-U and the
Wi-Fi are calling the each other's’ test methodologies faulty, so that they can find their own
preferred results. As mentioned above, both parties have begun to look closely on the FCC’s
recent comments regarding LTE-U/LAA collaboration.
Just to show the difference between LTE-U and the Wi-Fi testing that has been done so
far, and what different companies have concluded based on the testing results. We will look at
what Qualcomm has said regarding LTE-U.
Qualcomm (most frequently cited proponent of LTE-U):
Qualcomm has done an array of testing/presentations using eight Wi-Fi routers and then
eventually changing nodes between Wi-Fi/ LTE-U, and found that, generally, the performance of
Wi-Fi existence of LTE-U was somewhat better than in the existence of another Wi-Fi node. The
finding showed the finding from these testing have been presented by the company in a number
of forums. As the presentation at the ISART conference done by Yongbin Wei, senior director of
engineering for Qualcomm Technologies, demonstrates for both LTE-U and LAA.
In a test results submitted to the FCC, Qualcomm content it is not fair to compare Wi-Fi’s
performance in a noise free environment in order to measure the exist of LTE-U, which was
done in some testing. Qualcomm argues that a more fair comparison would be to test Wi-Fi’s
performance in an existence of other Wi-Fi nodes, in a both a noise free environment/exist of
other and in an existence of Wi-Fi node, As throughput tends to drop carelessly when compared
to a single Wi-Fi device operating in a clear channel(Shown in figure 5 ). For this project we will
not dive any further in what Google or Broadcom had to say about LTE-U/LAA as it is not
within our project scope.
Here is how this issue correlates to our project. If 4G LTE radios will operate to share 5
GHz frequencies with Wi-Fi devices, then they are subjected to the same testing for equal
sharing of those frequency channels that two Wi-Fi access points are subjected to. Our project
will determine whether multi-vendor channel sharing presents realistic problems or not.
.
5. 4
Introduction to Our Project
For our project, we have worked with the concept of access point channel sharing. Access
points, or wireless routers, are devices that connect multiple devices to a network through an
authentication phase and association phase, and then direct those devices on the best way to
interact on the channel that they are on. A channel is a frequency on the Wi-Fi spread spectrum-
separated by the 2.4 GHz and 5GHz band-that an access point can operate in. We have been
given a project to work with access points and devices to develop a better understanding of how
they interact within these elements. We will be conducting testing within these elements, and all
of our testing will be done using two of these access points at a time.
Test Objectives/ Goals
We wanted to find if unequal bandwidth sharing occurs between access points on the
same channel, this plays a key role in almost any environment where there is multiple Wi-Fi
networks present, especially places with a high density of access points and networks, this is
where many shared channels can occur.
Throughout our testing we were looking to see if the connection properties differ with
different vendor access points vs. same vendor access points when sharing channels; we also
wanted to find out whether different chipsets between the multiple access points affect the way
bandwidth is propagated. This was an important concept to take note of because if different
6. 5
chipsets react differently in an environment it may not a network to function to its maximum
capacity or performance. We tested in both 2.4 GHz and 5 GHz to find if channel sharing has a
different effect on throughput testing in the 2.4 GHz rather than in the 5 GHz band. We wanted
to take note on how the different bands reacted when it came to channel sharing, different bands,
vendors and chipsets. We were determined to find any consistent trends when testing for
upstream vs. downstream data transmission while sharing channels with same and different make
access points.
Items Used
Hardware
○ 2 Acer Aspire E15 Client Devices with 2x2 Wireless Adapter
○ 2 Dell Latitude E6400 Devices with Gigabit Ethernet Ports
○ 1 Netgear AC2350 Wireless Router
○ 2 ASUS RT-AC87U Wireless Routers
○ 2 Ethernet cords
Software
○ Xirrus Wi-Fi Analyzer
○ Chanalyzer Spectrum Analyzer
○ JPerf
Access Point Chipset Specs
Using information from WikiDevi.com, we were able to get the chipset information on the two
vendor routers:
● Netgear R7000
○ Uses only Broadcom chipset
● ASUS RT-AC87U
○ Uses Broadcom for 2.4 GHz and Quantenna chipset for 5GHz band
Test Methodology
For our testing, we first used Chanalyzer to determine the least crowded 2.4GHz channel
which was channel 1, and least crowded 5GHz channel which was channel 48. Once we found
what channel we were going to test on, we entered our 2 ASUS’s and 1 Netgear Access Point’s
web-based interface to configure routers (SSID, Channel, etc.)
7. 6
Later we set up two single access point client-server models: which consisted of two
laptops as servers and two laptops as clients. To test our signal strength we used Xirrus to test the
RSSI prior to each round of testing. Once everything was in place for us to run the test we made
sure that all testing was simultaneous for the 2 connections. We used JPerf to analyze
downstream and upstream for each access point and to see how the bandwidth was shared. We
ran 4 tests for upstream and 4 for downstream, with clients facing 4 different clockwise
directions at a close range and far range from router, with 60 second interval time (A total of 16
tests ran for each access point for 2.4 GHz and 5 GHz). The following charts represent the
average of east set of four tests. We combined the four results to get an average which is
displayed below.
Client-Server Model
We followed a client-server model where we had 2 consecutive test running at the same time.
For all downstream testing, one laptop was wired via ethernet as the Jperf client device and one
laptop was wirelessly linked to the access point as the Jperf server. When we did our upstream
testing, we flipped this setup so that the wireless laptop was the server while the client was the
wired laptop.
The Use of Jperf Throughput Testing Tool
8. 7
In order for us to perform throughput testing and document our findings, we used the
Jperf throughput testing tool. This software allows us to run the client-server model with the two
devices we planned to use. Jperf had various settings we could implement to document our
findings in the way we wanted to. The software allows each device to run in either the client
mode or server mode on a network, which fit our two device per router topology. We ran four
parallel streams, which is meant to maximize throughput performance. Port 5001 on the device
was used for the packet traffic to go out and in. The 60-second intervals were chosen for each
test session, and the output format was chosen to be in Megabits (Mbits). The report interval
would give us the throughput for each second of the test session, and these values would be
aggregated into the average value for the 60 seconds. In order to simulate a true network
environment, we used the TCP protocol for packet transmission.
2.4 GHz Pretest Findings
Same Vendor
10. 9
5 GHz Main Testing and Findings
For our testing, we first used Chanalyzer to determine the least crowded 5GHz channel
which was channel 48. Once we found what channel we were going to test on, we entered our 2
ASUS’s and 1 Netgear Access Point’s web-based interface to configure routers (SSID, Channel,
etc.)
Later we set up two single-access point client-server models: which consisted of two
laptops as servers and two laptops as clients. To test our signal strength we used Xirrus to test the
RSSI prior to each round of testing. Once everything was in place for us to run the test we made
11. 10
sure that all testing was simultaneous for the 2 connections. We used JPerf to analyze
downstream and upstream for each access point and to see how the bandwidth was shared. We
ran 4 tests for upstream and 4 for downstream, with clients facing 4 different clockwise
directions at a close range and far range from router, with 60 second interval time (A total of 16
tests ran for each access point).
Test 1 (Wireless Laptops Close To Router: SSID is -32 dBm)
15. 14
Simultaneous Downstream/Upstream Test Findings
Our Results and Analysis
After reviewing our findings, it can be concluded that same vendors having the same chipsets are
more reliable than different vendors, but even they can have a case where one access point is
operating using the majority of available bandwidth. We learned that there is no exact science to
justify how the bandwidth is allocated for the throughput test, but there were fewer fluctuations
when we were using the two ASUS routers that both use Broadcom. However, when we paired
16. 15
up the Netgear router and the ASUS router, which use different 5GHz Chipsets, there were many
instances where upstream and downstream throughput performance was heavily tilted towards
one connection. We concurred that because this occurred specifically at the 5GHz band, current
chipset technology may not be as adept at channel sharing at 5GHz as they are at 2.4GHz. The
upstream or downstream performance can vary greatly and fluctuate between the two connection
readings in any given round of testing. The only trend we found was that for 5GHZ testing, there
was a consistent tilt of the scales in the throughput performance towards one of the two
connections. This showed us that even though neither same chipset nor different chipset access
points channel sharing is viable for trying to attain equal throughput, access points with the same
chipset do not fluctuate in their bandwidth allocation that often. (You can look at this
phenomenon as the lesser of two evils)
Errors and Constraints within Our Testing Process
There were many errors and challenges that we faced during the course of our testing.
First, the wireless laptop’s Jperf results varied at times from the wired laptop’s, by about 0.1
Mbits/s throughput difference. We did not have an explanation for this phenomenon, and we
even mentioned it in our screencast. We were advised to go with the wired laptop’s results
during such instances. Another issue we faced was that the Dell laptops that we used as the wired
devices were very outdated laptops. It was not easy to maneuver within those systems to use the
software we needed to use, but we were able to mitigate any major issues or concerns. When
using Jperf, we needed port 5001 to be open on the laptop firewalls, and had to manually set
those permissions. The Acer wireless laptops were newer technology, but we had to disable the
McAfee firewalls on those devices in order to complete our testing. Another issue we found was
that one of the ASUS routers needed to be reset when we plugged it in and attempted to load the
web interface. The router had been previously used for other testing purposes, so we had to reset
it to default settings before conducting our testing. Of course, with any manual testing process,
there was the case of human error. In order for the testing to be 100% accurate, we needed to
start both 60-second connection sessions with the two access points simultaneously, and if we
started them even a fraction of a second apart, it could affect our findings. Finally, there was the
issue of time constraint. We were made aware of the simultaneous testing option very late in our
process in completing this project, and so we were unable to do a different vendor testing on
simultaneous mode.
Conclusion
Our project demonstrated how multiple access points within proximity of each other
share a channel that they are both configured to operate on. Our research and testing shows that
when a device is sharing a channel with another device, the throughput performance decreases.
From our testing we found that our hypothesis was true. When we used two access points from
the same vendor, our results showed that the devices had a fairly equal throughput reading more
often than not. When we used two access points from different vendors, our results showed that
17. 16
the devices did not have equal readings very often, since one device had higher throughput
readings compared to the average from the readings for same access points, but the other device
was significantly below average. However, we conclude that whether or not the two access
points sharing the 5GHz channel are from the same vendor or different vendor, the available
bandwidth can be unevenly shared. There is no clear pattern on how the channel bandwidth is
allocated by the access points. (Example in 5GHz, one client was at a throughput of 60 Mbit/s
while the other client was at 220 Mbits/s). We did see trends, however, we believe they are not
prominent enough to make any conclusions other than that there is no clear pattern on how the
bandwidth is shared by the Access Points.
Solution/Recommendations
The point that should be taken away from this is that if devices on a WI-FI network cannot even
get equal throughput with any consistency, how would devices connected to LTE-U services be
able to fairly share a 5GHz channel? Our recommendation is that devices on the 5GHz band are
not currently suited for access points from different vendors to share the unlicensed frequencies.
Even though there are no strict limitations that can prevent this from happening in terms of a
legal sense, this has to be set aside in the practical sense because of the fundamental
inconsistencies and unequal sharing that different vendors at 5GHz do.
