RLinWiFi: Contention Window Optimization in IEEE 802.11ax Networks with Deep Reinforcement Learning

1. Contention Window Optimization in IEEE 802.11ax Networks with Deep Reinforcement Learning Witold Wydmański and Szymon Szott, AGH University of Science and Technology

2. What is the reinforcement learning?

3. Machine learning paradigm Supervised model Observation Output Expected output Loss function 0 - Mole 1 - Melanoma https://www.kdnuggets.com/2020/05/deep-reinforcement-learning.html https://chesspathways.com/chess-openings/queens-pawn-opening/queens-gambit/

4. Basics of reinforcement learning https://www.funstockretro.co.uk/news/atari-pong-is-45-years-old-today/ https://keon.io/deep-q-learning/

5. Reinforcement learning paradigm https://www.kdnuggets.com/2020/05/deep-reinforcement-learning.html https://chesspathways.com/chess-openings/queens-pawn-opening/queens-gambit/ Reinforcement model Observation Action Reward function C6 You lost the game

6. CCOD: Reinforcement Learning in 802.11

7. CW optimization 15 DIFS DIFS 1 Transmission Delayed access Draft and backoff 8 DIFS 15 Frame CW max CW min CW Draft range 0 15 1023

8. CW optimization 15 CW max CW min CW Draft range 0 15 1023 DIFS DIFS 4 Collision 8 DIFS DIFS 15 DIFS 10 4 Collision Delayed access Draft and backoff Frame

9. Modeling an optimization problem Agent (player) Environment Observation Action Reward

10. CW optimization as an MDP https://menaentrepreneur.org/2017/02/importance-network-traffic-analysis/ Speedometer by https://www.flaticon.com/authors/vectors-market Agent (access point) Observed collision rate Observation Reward Throughput Action CW value

11. CCOD: Centralized contention window optimization with DRL

12. CCOD training architecture ns3-gym Scenario settings Python runtime Agent definition Teacher Agent https://www.nsnam.org/

13. CCOD results Static topology Dynamic topology Up to 40% throughput improvement

14. Thank you for attention

Editor's Notes

CCOD’s performance was evaluated in two topologies: static and dynamic. In the first one, the number of the stations connected to the network didn’t change over time. In the dynamic topology, the number of the stations was steadily increasing from 5 up to the goal number. Two first graphs show how the mean network throughput changed with varying number of stations. The third one shows the momentarily throughput in the dynamic topology, in which the simulation starts with 5 stations and ends with 50. Take a note on how quickly the performance of the stock 802.11ax CW control mechanism detoriates with the increasing number of stations. With 50 devices competing for the medium, the CCOD noted an improvement in throughput of over 40%
CCOD is trained

RLinWiFi: Contention Window Optimization in IEEE 802.11ax Networks with Deep Reinforcement Learning

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

RLinWiFi: Contention Window Optimization in IEEE 802.11ax Networks with Deep Reinforcement Learning

Editor's Notes