The document discusses the optimization of network throughput in IoT using deep reinforcement learning (DRL), highlighting key concepts such as Markov Decision Processes (MDP) and channel selection strategies. It covers various aspects of wireless communication including energy harvesting and medium access control optimization through techniques like contention window adjustment. The findings suggest promising results in improving wireless network performance through the application of DRL methods.

1. RLinWiFi –
optimizing network
thorughput with
Deep Reinforcement
Learning
Witold Wydmański, AGH

2. The bottleneck
Sources:
freepik.com
pixabay.com

3. Presentation plan
 Reinforcement learning in IoT
 General benefits in wireless communication
 A small trick in our sleeves

4. IoT. Wired networks – channel access
problem
Sources:
https://www.itrelease.com/2019/06/what-is-bus-topology-with-example/
http://spoat.jebrp.proe.hendil.mohammedshrine.org/basic-lan-wiring-diagram.html
Router

5. IoT. Wireless networks – channel
separation
2.4 GHz

6. IoT. Wireless networks – channel
separation
2.4GHz 2.5GHz 2.4GHz 2.5GHz

7. Markov Decision Process
MDP elements Tic-tac-toe Channel selection
State
Action
Reward
MDP elements Tic-tac-toe Channel selection
State Board
Action Drawing a sign
Reward Win
MDP elements Tic-tac-toe Channel selection
State Board Previous attempts
Action Drawing a sign Choosing a channel
Reward Win Lack of collision
Sources:
https://www.zabkat.com/blog/tic-tac-toe-learning.htm
http://ashleywang.me/machine-learning/2017/06/05/RL-Tictactoe.html

8. Deep Q-network
Sources:
https://medium.com/@carsten.friedrich/part-3-tabular-q-learning-a-tic-tac-toe-player-that-gets-better-and-better-fa4da4b0892a
2.4GHz
2.5GHz

9. IoT. Wireless networks – channel
separation results

10. IoT. Energy Harvesting
Sources:
https://anilkrpandey.wordpress.com/2020/02/09/energy-harvesting-and-wireless-power-transfer-antenna-for-iot-applications/

11. IoT. Energy conservation
Sources:
Energy-Efficient IoT Sensor Calibration with Deep Reinforcement Learning, A. ASHIQUZZAMAN et al.
Reinforcement learning based multi-access control and battery prediction with energy harvesting in IoT systems

12. WiFi. Medium Access Control
Sources:
https://blogs.arubanetworks.com/industries/real-world-examples-and-discussion-around-wifi-channels/

13. WiFi. Distributed Coordination Function
DIFS
DIFS 1 Transmission
Delayed access
Draft and
backoff
8
DIFS 15
Frame
CW max
CW min
CW
Draft range
0 15 1023

14. WiFi. Reinforcement Learning for CW
optimization
Sources:
Contention Window Optimization in IEEE 802.11ax Networks with Deep Reinforcement Learning – W. Wydmański, S. Szott
https://www.freecodecamp.org/news/an-intro-to-advantage-actor-critic-methods-lets-play-sonic-the-hedgehog-86d6240171d/
MDP elements CCOD
State Observed collision probabililty
Action Choosing a CW value
Reward Throughput
CW max
CW min
CW
0 15 1023

15. RLinWiFi. Centralized Contention
window Optimization with DRL

16. RLinWiFi. Results
Basic scenario Convergence scenario

17. The trick
Sources:
https://www.nextplatform.com/2017/03/21/can-fpgas-beat-gpus-accelerating-next-generation-deep-learning/

18. Conclusions