SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
Smart Congestion Control for Delay- and Disruption Tolerant Networks
Smart Congestion Control for Delay- and Disruption Tolerant Networks
1.
1/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Smart Congestion Control for Delay- and
Disruption Tolerant Networks
Aloizio P. Silva14 Katia Obraczka2 Scott Burleigh3
Celso M. Hirata1
1Instituto Tecnológico de Aeronáutica - ITA
Department of Electronic and Computer Engineering
2University of California Santa Cruz - UCSC
Department of Computer Engineering
3Jet Propulsion Laboratory - NASA
California Institute of Technology - Caltech
4Universidade Federal de Minas Gerais - UFMG
Department of Computer Science
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
2.
2/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Outline
1 Introduction
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
2 Smart-DTN-CC
Reinforcement learning
Q-learning
3 Experimental Analysis
4 Conclusion
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
3.
3/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Delay and Disruption Tolerant Networks - DTNs
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
4.
4/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Delay and Disruption Tolerant Networks - DTNs
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
5.
5/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Delay and Disruption Tolerant Networks - DTNs
Extreme environments
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
6.
6/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Delay and Disruption Tolerant Networks - DTNs
Original Application
Deep space communication
IPN - Interplanetary Networks
Characteristics
Long and variable delays
Intermittent connectivity
Asymmetric data rates
High error rates
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
7.
7/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Congestion Problem
Incoming traffic rate exceeds the rate at which packets are
processed.
Network becomes saturated/ overloaded.
Figure: Congestion in networks
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
8.
8/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
DTN Congestion Problem
TCP congestion control mechanisms don’t work in DTN’s.
Figure: DTN congestion
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
9.
9/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
Existing DTN Congestion Control Mechanism
Main features:
DTN application specific
Network’s global information
Reactive approach
Dependence of the routing protocol
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
10.
10/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Delay and Disruption Tolerant Network - DTN
Congestion Problem
The State-of-the-Art DTN Congestion Control
A Novel DTN Congestion Control Framework
DTN congestion control requirements:
1 Local and autonomous
2 Proactive and reactive
3 Able to adapt automatically to network dynamics
Our solution: machine learning.
1 Supervised learning
2 Unsupervised learning
3 Reinforcement learning
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
11.
11/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Reinforcement learning
Employ reinforcement learning
Objective: perform congestion control
Need to make a sequence of good decisions to mitigate
DTN congestion
Similar to training a pet dog
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
12.
12/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Reinforcement learning
Figure: Reiforcement learning
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
13.
13/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Smart-DTN-CC
Figure: DTN node state machine
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
14.
14/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Smart-DTN-CC
Figure: DTN node actions
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
15.
15/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Smart-DTN-CC
Action selection methods
1 Boltzmann distribution
Parameter: a temperature T controls the amount of
exploration.
2 WoLF: Win or Learn Fast
Parameter: two learning rates: γmin and γmax .
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
16.
16/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Smart-DTN-CC
Figure: Reward function
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
17.
17/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Reinforcement learning
Q-learning
Q-learning Algorithm
Figure: Q-learning
algorithm Figure: Q-values update function
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
18.
18/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Experimental Methodology
We use the ONE Simulator
Evaluation Metrics:
Average delivery ratio
Average end-to-end latency
The scenario is a terrestrial mobile network
Mobility models: RW, RWP and SPMBM.
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
19.
19/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Experimental Methodology
We evaluate SMART-DNT-CC by itself
We compare SMART-DTN-CC’s performance against;
1 AFNER
2 CCC
3 RRCC
4 SR
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
20.
20/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Results: Average cumulative reward as a function of the
simulation time
Figure: Boltzmann action selection method
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
21.
21/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Results: Average delivery ratio as a function of buffer
threshold.
(a) Delivery Ratio
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
22.
22/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Results: Comparative analysis
Average delivery ratio as a function of message generation
period.
(b) Terrestrial Scenario
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
23.
23/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Results: Comparative analysis
Average delivery ratio for different mobility models and
routing protocol (Buffer size of 500kB, Message Generation
Period of 300 s, Buffer Threshold for Smart-DTN-CC of
60%)
(c) Random Walk (d) Random Way Point (e) Shortest Path Map
Based Movement
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
24.
24/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Results: Comparative analysis
Average latency for different mobility models and routing
protocol (Buffer size of 500kB, Message Generation Period
of 300 s, Buffer Threshold for Smart-DTN-CC of 60%)
(f) Random Walk (g) Random Way Point (h) Shortest Path Map
Based Movement
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs
25.
25/ 36
Introduction
Smart-DTN-CC
Experimental Analysis
Conclusion
Conclusion
We investigate the congestion control in DTN.
A novel congestion control framework was designed
(SMART-DTN-CC):
It is based on machine learning: reinforcement learning
it allows the DTN nodes autonomously mitigate congestion
It adopts an hybrid approach: proactive and reactive, open-
and closed loop.
it allows a DTN node to adapt to different DTN
environments
The results shown that it outperforms the existing
mechanisms.
IEEE SECON’2016 - London, UK - June 27-30, 2016 Smart Congestion Control for DTNs