Smart Congestion Control for Delay- and Disruption Tolerant Networks
1. 1/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Smart Congestion Control for Delay- and
Disruption Tolerant Networks
Aloizio P. Silva14 Katia Obraczka2 Scott Burleigh3
Celso M. Hirata1
1Instituto Tecnológico de Aeronáutica - ITA
Department of Electronic and Computer Engineering
2University of California Santa Cruz - UCSC
Department of Computer Engineering
3Jet Propulsion Laboratory - NASA
California Institute of Technology - Caltech
4Universidade Federal de Minas Gerais - UFMG
Department of Computer Science
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
2. 2/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Outline
1 Introduction
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
2 Smart-DTN-CC
Reinforcement learning
Q-learning
3 Experimental Analysis
4 Conclusion
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
3. 3/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Delay and Disruption Tolerant Networks - DTNs
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
4. 4/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Delay and Disruption Tolerant Networks - DTNs
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
5. 5/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Delay and Disruption Tolerant Networks - DTNs
Extreme environments
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
6. 6/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Delay and Disruption Tolerant Networks - DTNs
Original Application
Deep space communication
IPN - Interplanetary Networks
Characteristics
Long and variable delays
Intermittent connectivity
Asymmetric data rates
High error rates
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
7. 7/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Congestion Problem
Incoming traffic rate exceeds the rate at which packets are
processed.
Network becomes saturated/ overloaded.
Figure: Congestion in networks
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
8. 8/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
DTN Congestion Problem
TCP congestion control mechanisms don’t work in DTN’s.
Figure: DTN congestion
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
9. 9/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Existing DTN Congestion Control Mechanism
Main features:
DTN application specific
Network’s global information
Reactive approach
Dependence of the routing protocol
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
10. 10/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
A Novel DTN Congestion Control Framework
DTN congestion control requirements:
1 Local and autonomous
2 Proactive and reactive
3 Able to adapt automatically to network dynamics
Our solution: machine learning.
1 Supervised learning
2 Unsupervised learning
3 Reinforcement learning
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
11. 11/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Reinforcement learning
Employ reinforcement learning
Objective: perform congestion control
Need to make a sequence of good decisions to mitigate
DTN congestion
Similar to training a pet dog
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
15. 15/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Smart-DTN-CC
Action selection methods
1 Boltzmann distribution
Parameter: a temperature T controls the amount of
exploration.
2 WoLF: Win or Learn Fast
Parameter: two learning rates: γmin and γmax .
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
18. 18/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Experimental Methodology
We use the ONE Simulator
Evaluation Metrics:
Average delivery ratio
Average end-to-end latency
The scenario is a terrestrial mobile network
Mobility models: RW, RWP and SPMBM.
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
23. 23/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Results: Comparative analysis
Average delivery ratio for different mobility models and
routing protocol (Buffer size of 500kB, Message Generation
Period of 300 s, Buffer Threshold for Smart-DTN-CC of
60%)
(c) Random Walk (d) Random Way Point (e) Shortest Path Map
Based Movement
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
24. 24/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Results: Comparative analysis
Average latency for different mobility models and routing
protocol (Buffer size of 500kB, Message Generation Period
of 300 s, Buffer Threshold for Smart-DTN-CC of 60%)
(f) Random Walk (g) Random Way Point (h) Shortest Path Map
Based Movement
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
25. 25/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Conclusion
We investigate the congestion control in DTN.
A novel congestion control framework was designed
(SMART-DTN-CC):
It is based on machine learning: reinforcement learning
it allows the DTN nodes autonomously mitigate congestion
It adopts an hybrid approach: proactive and reactive, open-
and closed loop.
it allows a DTN node to adapt to different DTN
environments
The results shown that it outperforms the existing
mechanisms.
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs