Successfully reported this slideshow.

Smart Congestion Control for Delay- and Disruption Tolerant Networks

1

Share

1 of 25
1 of 25

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Smart Congestion Control for Delay- and Disruption Tolerant Networks

  1. 1. 1/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Smart Congestion Control for Delay- and Disruption Tolerant Networks Aloizio P. Silva14 Katia Obraczka2 Scott Burleigh3 Celso M. Hirata1 1Instituto Tecnológico de Aeronáutica - ITA Department of Electronic and Computer Engineering 2University of California Santa Cruz - UCSC Department of Computer Engineering 3Jet Propulsion Laboratory - NASA California Institute of Technology - Caltech 4Universidade Federal de Minas Gerais - UFMG Department of Computer Science IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  2. 2. 2/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Outline 1 Introduction Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control 2 Smart-DTN-CC Reinforcement learning Q-learning 3 Experimental Analysis 4 Conclusion IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  3. 3. 3/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control Delay and Disruption Tolerant Networks - DTNs IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  4. 4. 4/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control Delay and Disruption Tolerant Networks - DTNs IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  5. 5. 5/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control Delay and Disruption Tolerant Networks - DTNs Extreme environments IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  6. 6. 6/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control Delay and Disruption Tolerant Networks - DTNs Original Application Deep space communication IPN - Interplanetary Networks Characteristics Long and variable delays Intermittent connectivity Asymmetric data rates High error rates IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  7. 7. 7/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control Congestion Problem Incoming traffic rate exceeds the rate at which packets are processed. Network becomes saturated/ overloaded. Figure: Congestion in networks IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  8. 8. 8/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control DTN Congestion Problem TCP congestion control mechanisms don’t work in DTN’s. Figure: DTN congestion IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  9. 9. 9/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control Existing DTN Congestion Control Mechanism Main features: DTN application specific Network’s global information Reactive approach Dependence of the routing protocol IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  10. 10. 10/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Delay and Disruption Tolerant Network - DTN Congestion Problem The State-of-the-Art DTN Congestion Control A Novel DTN Congestion Control Framework DTN congestion control requirements: 1 Local and autonomous 2 Proactive and reactive 3 Able to adapt automatically to network dynamics Our solution: machine learning. 1 Supervised learning 2 Unsupervised learning 3 Reinforcement learning IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  11. 11. 11/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Reinforcement learning Q-learning Reinforcement learning Employ reinforcement learning Objective: perform congestion control Need to make a sequence of good decisions to mitigate DTN congestion Similar to training a pet dog IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  12. 12. 12/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Reinforcement learning Q-learning Reinforcement learning Figure: Reiforcement learning IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  13. 13. 13/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Reinforcement learning Q-learning Smart-DTN-CC Figure: DTN node state machine IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  14. 14. 14/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Reinforcement learning Q-learning Smart-DTN-CC Figure: DTN node actions IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  15. 15. 15/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Reinforcement learning Q-learning Smart-DTN-CC Action selection methods 1 Boltzmann distribution Parameter: a temperature T controls the amount of exploration. 2 WoLF: Win or Learn Fast Parameter: two learning rates: γmin and γmax . IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  16. 16. 16/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Reinforcement learning Q-learning Smart-DTN-CC Figure: Reward function IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  17. 17. 17/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Reinforcement learning Q-learning Q-learning Algorithm Figure: Q-learning algorithm Figure: Q-values update function IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  18. 18. 18/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Experimental Methodology We use the ONE Simulator Evaluation Metrics: Average delivery ratio Average end-to-end latency The scenario is a terrestrial mobile network Mobility models: RW, RWP and SPMBM. IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  19. 19. 19/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Experimental Methodology We evaluate SMART-DNT-CC by itself We compare SMART-DTN-CC’s performance against; 1 AFNER 2 CCC 3 RRCC 4 SR IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  20. 20. 20/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Results: Average cumulative reward as a function of the simulation time Figure: Boltzmann action selection method IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  21. 21. 21/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Results: Average delivery ratio as a function of buffer threshold. (a) Delivery Ratio IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  22. 22. 22/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Results: Comparative analysis Average delivery ratio as a function of message generation period. (b) Terrestrial Scenario IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  23. 23. 23/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Results: Comparative analysis Average delivery ratio for different mobility models and routing protocol (Buffer size of 500kB, Message Generation Period of 300 s, Buffer Threshold for Smart-DTN-CC of 60%) (c) Random Walk (d) Random Way Point (e) Shortest Path Map Based Movement IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  24. 24. 24/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Results: Comparative analysis Average latency for different mobility models and routing protocol (Buffer size of 500kB, Message Generation Period of 300 s, Buffer Threshold for Smart-DTN-CC of 60%) (f) Random Walk (g) Random Way Point (h) Shortest Path Map Based Movement IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
  25. 25. 25/ 36 Introduction Smart-DTN-CC Experimental Analysis Conclusion Conclusion We investigate the congestion control in DTN. A novel congestion control framework was designed (SMART-DTN-CC): It is based on machine learning: reinforcement learning it allows the DTN nodes autonomously mitigate congestion It adopts an hybrid approach: proactive and reactive, open- and closed loop. it allows a DTN node to adapt to different DTN environments The results shown that it outperforms the existing mechanisms. IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs

×