This document summarizes a B.Tech project thesis defense on distributed learning and adaptation in cognitive radio. The project aimed to develop a decentralized learning system to map secondary users to idle primary network channels without collision. The methodology used a novel Distributed Learning and Adaptive algorithm (DALA) policy based on stochastic learning and an Upper Confidence Bound algorithm. Simulation results showed the DALA policy converged channel selection probabilities and secondary users were assigned different channels. Mini-batch stochastic learning improved performance over DALA by reducing regret. Thresholding channel probabilities provided faster convergence. Future work may include variable step sizes, dynamic channel availability, and new ranking schemes.

1. Distributed Learning and Adaptation in Cognitive Radio
Department of Electronics and Communication Engineering
B.Tech Project Thesis Defense
May 05th, 2017
Supervisor: Submitted by:
Dr. P.M. Pradhan Yash Gangrade
Assistant Professor Shubham Goyal
Indian Institute of Technology Roorkee

2. Outline
o Motivation
o Problem Statement
o Introduction
o Methodology and Solutions
o Simulation Results
o Conclusion
o Future Scope of Research
2

3. 3
o Although the advances in wireless communications technology have made the spectrum
utilization techniques sophisticated, there are a number of gaps in their applications that
needs to be filled.
Motivation
Fig. 1) CISCO Forecast for the predicted number of radio devices by 2021

4. o Motivation and Scope 
o Problem Statement
o Introduction
o Methodology and Solutions
o Simulation Results
o Conclusion
o Future Scope of Research
4

5. 5Problem Statement
o Develop a decentralized learning and adaptation system to map secondary users to the idle
channels of the primary network.
o Select different channel for each user to ensure that no collision happens.
o Select most available channels for secondary users according to a ranking index vector.

6. o Motivation and Scope 
o Problem Statement 
o Introduction
o Methodology and Solutions
o Simulation Results
o Conclusion
o Future Scope of Research
6

7. 7
o Cognitive radio is an evolutionary modern approach to deal with radio network in the field of
wireless communications.
o The CR networks contain the capability of detecting, monitoring, interpreting and sensing its
surrounding and thereby dynamically reconfiguring its own operating parameters.
o This reconfiguration help in maximizing throughput, reduce the interference issue, facilitate
interoperability, and access the secondary user markets etc.
o Primary User and Secondary User
Cognitive Radio

8. 8Cognitive Radio Network in Action
Fig. 1) Basic Working of Cognitive Radio (source: xgtechnology)

9. 9
o It is a promising and propitious technology in cognitive radio to alleviate the problem of
spectrum scarcity and assist in increasing the utilization of spectrum.
o It is a spectrum sharing paradigm that gives secondary users power to access the abundant
spectrum white spaces, holes in the primary traffic bands or licensed spectrum bands.
Dynamic Spectrum Access (DSA)

10. 10
• Stochastic Learning is characterized as a group of various learning algorithms which executes
an expansive dataset by the consecutive handling of arbitrary examples of the dataset.
• Every step is also a deterministic step, means operating parameters are updated at each step.
Stochastic Learning is exceptionally viable for dealing with enormous dataset and frameworks.
• In stochastic gradient descent operating parameters are updated at each stochastic step unlike
gradient decent where operating parameters are updated after executing the complete
dataset.
Stochastic Learning

11. o Motivation and Scope 
o Problem Statement 
o Introduction 
o Methodology and Solutions
o Simulation Results
o Conclusion
o Future Scope of Research
11

12. 12
• A cognitive radio model was developed which consists of N primary channels and M secondary
users
• Since secondary users might be in the close range so, we are also considering the possibility of
SUs interfering with each other while selecting idle primary channel simultaneously.
• The scenario of broadband spectrum access was investigated. In this case, a large number of
primary channels is available. M<N is considered.
• The behaviour of every primary channel i.e. Xi(n) is formulated as a random Bernoulli process
for our network model.
• Xi(n) = 1; Channel i is available
0; otherwise
Network Model

13. 13
• First policy proposed for the efficient spectrum utilization is the novel DALA policy.
• This DALA policy is based on two underlying distributed learning and adaptation algorithms
namely, SLA-based learning algorithm and Upper Confidential Bound (UCB1) algorithm.
• The DALA policy can be studied in two parts, first section is about primary channel sensing and
second section is about the Distributed and Stochastic Access.
Distributed Learning and Adaptive Learning (DALA)
Policy

14. 14
o Different channel mean availabilities are considered and each SU desires to select the most
available or probable channel to achieve throughput.
o The mean channel availability vector i.e. Ө vector of channels not known. Thus, a good sensing
policy is a necessity.
o The problem is modelled as a single player Classical Multi-Armed Bandit (MAB) problem for N
channels (arms). Also, we are using a policy called UCB1 which is an optimal-learning learning
algorithm to compute the unknown Өi’s.
I) Distributed Learning and Primary Channels Sensing

15. 15
o UCB1 is an index-based learning algorithm which computes the parameter to rank the arms of
the MAB problem.
o The SU selects the highest-ranked channel at each nth time slot. The reward process in each of
the arm but it is at the same time, also a random process.
I) Distributed Learning and Primary Channels Sensing
�Өi
j (Ti
j (n)) ≜ (Ti
j (n))-1Xi
j(n)

16. 16
o Ti
j (n) : total times that secondary user j selects a primary channel i till the time slot n.
o Xi
j(n): total number of times till the nth time slot is when secondary user j senses channel i and
detects no collision
o Ii
j (n): ranking-index vector for each channel i (based on availability) for every SU j using
extended UCB1 algorithm.
o Then, we selects on of the channels whose ranking index is in first M highest in Ii
j (n) to access.
Let we select kth-highest then 0 ≤ k ≤ M. UCB1 algorithm first prefers arms which are never
sensed before over which has been sensed before.
I) Distributed Learning and Primary Channels Sensing

17. 17II) Distributed Access and Stochastic Access
a) Channel Selection

18. 18II) Distributed Access and Stochastic Access
b) Channel Selection Probability Update Rule

19. 19
o A mini-batch learning scheme is proposed to improve the time complexity for channel
selection probability convergence and performance improvement of DALA policy.
o Instead of updating parameter at each step like stochastic learning automata, we accumulate
the channel selection probability update factor over a sequence of mini-batch and update it at
the end of a mini-batch in deterministic step.
o Mini-batch size is the no of consecutive stochastic steps taken before a deterministic step, b.
Mini Batch SLA

20. 20
o Regret is a common metric used to measure the throughput loss of a given access policy
under learning. Normalized regret is used.
o Defined as the difference of throughput between the ideal scenario and a given policy. Ideal
scenario is the case when Ө is known to the SU.
Regret as a Performance Metric

21. o Motivation and Scope 
o Problem Statement 
o Introduction 
o Methodology and Solutions 
o Simulation Results
o Conclusion
o Future Scope of Research
21

22. 22
I) DALA Policy – Convergence of Channel Selection Probability
N M Case Mean Channel Availabilities (Ө)
9 4 1 [0.1, 0.2, 0.3.........0.9]
2 [0.25, 0.38, 0.46, 0.54, 0.6, 0.68, 0.92, 0.12, 0.85]
3 [0.51, 0.52, 0.53.........0.59]
o Simulated DALA policy for three different cases with the following operating parameters.
o Mean channel availability step size, b = 0.01
o No. of iterations or time slots = 5000

23. 23
Case I – Channel Selection Probability vs Time Slot for SU 1
(b=0.01)

24. 24
Case I – Channel Selection Probability vs Time Slot for SU 2
(b=0.01)

25. 25
Case I – Channel Selection Probability vs Time Slot for SU 3
(b=0.01)

26. 26
Case I – Channel Selection Probability vs Time Slot for SU 4
(b=0.01)

27. 27
Case III – Channel Selection Probability vs Time Slot for SU 1
(b=0.01)

28. 28
Case III – Channel Selection Probability vs Time Slot for SU 3
(b=0.01)

29. 29
Case III – Channel Selection Probability vs Time Slot for SU 1
(b=0.001)

30. 30
Case III – Channel Selection Probability vs Time Slot for SU 4
(b=0.001)

31. 31I) DALA Policy Output – Convergence of Channel Selection
Probability
o The channel convergence results obtained for each SU using DALA policy are shown below for
Case 1 (b=0.01) and Case 3 (b=0.001) shown respectively;
N M Secondary User (j) Channel selected by corresponding user (i)
9 4 1 6
2 7
3 8
4 9
N M Secondary User (j) Channel selected by corresponding user (i)
9 4 1 6
2 7
3 9
4 8

32. 32II) Mini Batch SLA – Variant of Learning Automation
o Batch Size of 10 was considered for implementing the Mini Batch SLA Results.
o The following table shows the output results of a simulation run of Mini Batch SLA applied on
the mean channel availability vector of Case 1;
o The output table is as follows:
N M Case Mean Channel Availabilities (Ө)
9 4 1 [0.1, 0.2, 0.3.........0.9]
N M Secondary User (j) Channel selected by corresponding user (i)
9 4 1 7
2 8
3 6
4 9

33. 33
Mini Batch SLA (Case 1) – Convergence Curve for SU 1

34. 34
Mini Batch SLA (Case 1) – Convergence Curve for SU 2

35. 35
Mini Batch SLA (Case 1) – Convergence Curve for SU 3

36. 36
Mini Batch SLA (Case 1) – Convergence Curve for SU 4

37. 37III) Regret as a performance metric
o In subsequent slides, regret comparison between the proposed policies are demonstrated
and analysed.
o It was found that Mini Batch SLA performs better than different cases of DALA policy in terms
of regret.

38. 38
Regret comparison between Case 1 of DALA policy (b=0.01) and
Mini Batch SLA (step size = 10)

39. 39
Regret comparison between Case 2 of DALA policy (b=0.01) and
Mini Batch SLA (step size = 10)

40. 40
Regret comparison between Case 3 of DALA policy (b=0.001) and
Mini Batch SLA (step size = 10)

41. 41IV) Thresholding
o A threshold level of 0.9 is considered to achieve the bridging of channel selection probability
with a faster convergence.
o As any channel selection probability for a SU crosses the threshold level (here 0.9), that
particular channel selection probability is converged to 1 and rest channel selection
probabilities are converged to 0.
o Performance in terms of regret was found to better with thresholding in all the cases.

42. 42
Regret comparison between Case 1 of DALA policy (b=0.01) with
and without threshold

43. 43
Regret comparison between Case 3 of DALA policy (b=0.001) with
and without threshold

44. 44
Regret comparison between Mini Batch SLA with and without
threshold

45. o Motivation and Scope 
o Problem Statement 
o Introduction 
o Methodology and Solutions 
o Simulation Results 
o Conclusion
o Future Scope of Research
45

46. 46
o The proposed DALA policy matches all the secondary users to different channels ensuring no
collision in transmission.
o All of the M SUs converge for M best channels which leads to high throughput of the system.
o When mean channel availabilities are very close (eg. 0.01 difference in values), then reducing
the step size of channel selection probability ensures a better convergence.
o Mini Batch SLA implemented and it demonstrated significant improvement in terms of regret
over the SLA of DALA policy.
o Thresholding was also introduced and it provided even better regret performance for all the
cases.
Conclusions

47. o Motivation and Scope 
o Problem Statement 
o Introduction 
o Methodology and Solutions 
o Simulation Results 
o Conclusion 
o Future Scope of Research
47

48. 48
o Variable and Dynamic Step Size
o Dynamic Mean Channel Availability and Multiple Arms of a SU
o Adopting different policies and models of learning and adaptation
o A new Ranking Scheme
Future Scope of Research

49. Thank You !
This presentation bears an imprint of many people,
we sincerely thank them for their support:
Dr. Anand Bulusu, Padmanabh Pande, and Paras Mehandiratta
In the new era, thoughts and ideas itself will be
transmitted by radio.
- Gueglilmo Marconi
“ “

50. Thank you!
Questions?
“ “