A new kind of machine learning technique and its applications to network management problems

1. By: Ali Mohammad Saghiri
Advisor: Prof. M. R. Meybodi

2. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
 Other works
 Conclusion
1

3. 252 /57
Cellular
Automata (CAs)
Learning
Automata (LAs)
CLAs DCLAs
CLAs with dynamic structure,
neighborhood,….
1/60

4. 252 /57
CADCLA
CADCLA
-VL
Management in
cognitive peer-
to-peer
networks
2/60
CLA

5. 252 /57
 Overview
 Background
◦ Learning Automaton (LA)
◦ Cellular Learning Automaton (CLA)
◦ Peer-to-Peer Networks
◦ Cognitive Networks
 Statement of the problem
 Dynamic models of CLAs
 Other Works
 Conclusion

6. 252 /57
Action is chosen based on
action probability vector
Environment
process the
input action
Environment
responds the
action by
reinforcement
Automaton
process the
input signal
Action probability vector
is updated using a
learning algorithm
(n)
Environment
Learning
Automaton
(n)
3/60

7. 252 /57
 Local rule
◦ takes the set of actions selected by the LAs in the local environment
◦ return a reinforcement signal
4/60
1
2
l
l
p
p
 
 
 
1
2
k
k
p
p
 
 
 
1
2
m
m
p
p
 
 
 
1
2
j
j
p
p
 
 
 
LAm
LAj
LAk
LAl
LAi
α
α
α
α
αα
αα
Local Rule
β
Local environment of celli

8. 252 /575/60
Self-organized capability
LACA
CLA
Decision making(learning) in
unknown environment
Designing self-organized
systems with learning capability
in unknown environment

9. 252 /57
Static CLAs
Dynamic CLAs
Open CLAs
Closed CLAs
Synchronous CLAs
Asynchronous CLAs
CLAs with one LA in each cell
CLAs with multiple LAs in
each cell
CLAs
6/60

10. 252 /57
 Peer-to-peer Network
 Challenges
◦ Large scale
◦ Dynamicity
 Management mechanism
◦ Self-organized
 ACO, GNG,…
 CLAs (DCLAs) [not reported]
4/407/60

11. 252 /57
 Definition:
◦ a cognitive network can perceive current network conditions,
and then plan, decide, and act on those conditions.
 Learning capability
 Machine learning methods
 Domains:
◦ Cognitive radio networks
◦ Cognitive mesh networks
◦ Cognitive sensor networks
◦ Cognitive peer-to-peer networks[not reported]
8/60

12. 252 /57
 Cognitive networks
◦ A three layer framework reported by Thomas [Well-Known]
9/60

13. 252 /57
 Overview
 Background
 Statement of the problem
◦ Theory of CLA
◦ Network management
 Dynamic models of CLAs
◦ CADCLA
◦ CADCLA-VL
 Conclusion
11/40

14. 252 /57
 Dynamicity Definitions
◦ How can we define DCLAs?
 Changes in the cellular structure
 Changes in the number of LAs of the cells
◦ Dynamicity Analysis
 Under which condition the DCLA is expedient?
10/60

15. 252 /57
 Peer-to-peer networks
◦ How we can design cognitive peer-to-peer networks?
 Self-organized
 Distributed Computation
 Adaptive
◦ How we can design CLA based management mechanisms?
 Which model of CLA or DCLA?
 New rules?
11/60

16. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
◦ CADCLA [Closed][Asynchronous][Dynamic]
◦ CADCLA-VL [varying number of LAs in each cell]
 Other Works
 Conclusion
1

17. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
◦ CADCLA
 Definition
 Behavior
 Application
◦ CADCLA-VL
 Other Works
 Conclusion
1

18. 252 /57
 𝐺 =< 𝑉, 𝐸 >
 A = 𝐿𝐴1, … , 𝐿𝐴 𝑛
 Ф is the set of actions
 𝛹 is the set of attributes
 𝐹1is the local rule
 𝐹2 is the structure
updating rule
10/40
LAi
LAk LAl
LAj
< Ф𝑖, 𝛹𝑖 >
< Ф 𝑘, 𝛹 𝑘 > < Ф𝑙, 𝛹𝑙 >
< Ф𝑗, 𝛹𝑗 >
12/60

19. 252 /57
 𝐺 =< 𝑉, 𝐸 >
 A = 𝐿𝐴1, … , 𝐿𝐴 𝑛
 Ф is the set of actions
 𝛹 is the set of attributes
 𝐹1 is the local rule
◦ 𝛽𝑖 is the reinforcement signal
◦ 𝜁𝑖 is the restructuring signal
 𝐹2 is the structure updating
rule
LAi
𝛹𝑖
𝛽𝑖 Action
Celli
Local Environment
Structure Updating Rule
𝑁𝑖
1
Ф𝑖, 𝑁𝑖
𝜁𝑖
Local Rule
𝛷𝑖 − 𝛷𝑖
1
12/60

20. 252 /5713/60
𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑒 𝑐𝑒𝑙𝑙𝑖
LAi
𝛹𝑖
𝛽𝑖 Action
Celli
Local Environment
Structure Updating Rule
𝑁𝑖
1
Ф𝑖, 𝑁𝑖
𝜁𝑖
Local Rule
𝛷𝑖 − 𝛷𝑖
1
Application determine which cell
must be activated

21. 252 /5710/60
𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑒 𝑐𝑒𝑙𝑙𝑖
LAi
𝛹𝑖
𝛽𝑖 Action
Celli
Local Environment
Structure Updating Rule
𝑁𝑖
1
Ф𝑖, 𝑁𝑖
𝜁𝑖
Local Rule
𝛷𝑖 − 𝛷𝑖
1
Preparation phase
Structure updating phase
State updating phase
13/60

22. 252 /5710/60
𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑒 𝑐𝑒𝑙𝑙𝑖
LAi
𝛹𝑖
𝛽𝑖 Action
Celli
Local Environment
Structure Updating Rule
𝑁𝑖
1
Ф𝑖, 𝑁𝑖
𝜁𝑖
Local Rule
𝛷𝑖 − 𝛷𝑖
1
Preparation phase
Structure updating phase
State updating phase
13/60

23. 252 /5710/60
𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑒 𝑐𝑒𝑙𝑙𝑖
LAi
𝛹𝑖
𝛽𝑖 Action
Celli
Local Environment
Structure Updating Rule
𝑁𝑖
1
Ф𝑖, 𝑁𝑖
𝜁𝑖
Local Rule
𝛷𝑖 − 𝛷𝑖
1
Preparation phase
Structure updating phase
State updating phase
13/60

24. 252 /5710/60
𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑒 𝑐𝑒𝑙𝑙𝑖
LAi
𝛹𝑖
𝛽𝑖 Action
Celli
Local Environment
Structure Updating Rule
𝑁𝑖
1
Ф𝑖, 𝑁𝑖
𝜁𝑖
Local Rule
𝛷𝑖 − 𝛷𝑖
1
Preparation phase
Structure updating phase
State updating phase
𝜁𝑖= 1
13/60

25. 252 /5710/60
𝑎𝑐𝑡𝑖𝑣𝑎𝑡𝑒 𝑐𝑒𝑙𝑙𝑖 𝜁𝑖= 1
Preparation phase
Structure updating phase
State updating phase
LAi
𝛹𝑖
𝛽𝑖 Action
Celli
Local Environment
Structure Updating Rule
𝑁𝑖
1
Ф𝑖, 𝑁𝑖
𝜁𝑖
Local Rule
𝛷𝑖 − 𝛷𝑖
1
13/60

26. 252 /57
 Entropy
◦ Analyzing the changes in the actions of the CLA
 Lower entropy → lower changes in the actions
 Restructuring tendency
◦ Analyzing the changes in the structure of the CLA
 Lower restructuring tendency → lower changes in the structure
T(k)=σ𝑖=1
𝑛
𝜁𝑖(𝑘)
   
1
N
i
i
H k H k

        
1
ln
im
i ij ij
j
H k p k p k

  
14/60

27. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
◦ CADCLA
 Definition
 Behavior
 Application
◦ CADCLA-VL
 Other Works
 Conclusion
1

28. 252 /57
 Expediency
◦ An CADCLA is expedient if
 in the long run, the LA of every cell receives more reward than
a pure-chance automaton
 Which set of conditions results in a expedient CADCLA?
◦ Under which learning algorithm for the LAs
 LRP learning algorithm
 LRεP learning algorithm
15/60
Conditions?
CLA is
Expedient

29. 252 /57
 𝑌(𝑡) 𝑡≥0
◦ 𝑌(𝑡) = 𝑁 𝑡 , P 𝑡 , Ф 𝑡
◦ N 𝑡 = N1 𝑡 , N2 𝑡 , … , N 𝑛 𝑡
T
 𝑁𝑖 = {𝑐𝑒𝑙𝑙𝑗 ∈ 𝑉|𝑑𝑖𝑠𝑡 𝑐𝑒𝑙𝑙𝑖, 𝑐𝑒𝑙𝑙 𝑗 < θi}
◦ 𝑃(𝑡) = P1 𝑡 , P2 𝑡 , … , Pn 𝑡
T
◦ Ф 𝑡 = Ф1 𝑡 , Ф2 𝑡 , … , Ф 𝑛 𝑡
T
 𝑑𝑖𝑗
𝛽
𝑌 𝑡
 the reward probability of action αj of LAi.
16/60

30. 252 /57
 𝑌(𝑡) 𝑡≥0
◦ 𝑌(𝑡) = 𝑁 𝑡 , P 𝑡 , Ф 𝑡
◦ N 𝑡 = N1 𝑡 , N2 𝑡 , … , N 𝑛 𝑡
T
◦ 𝑃(𝑡) = P1 𝑡 , P2 𝑡 , … , Pn 𝑡
T
 Pi 𝑡 = 𝑝𝑖1 𝑡 , 𝑝𝑖2 𝑡 , … , 𝑝𝑖𝑟 𝑡
T
◦ Ф 𝑡 = Ф1 𝑡 , Ф2 𝑡 , … , Ф 𝑛 𝑡
T
 𝑑𝑖𝑗
𝛽
𝑌 𝑡
 the reward probability of action αj of LAi.
23/60
𝑝𝑖1 𝑡
LAi
𝑝𝑖𝑗 𝑡
𝑝𝑖𝑟 𝑡
16/60

31. 252 /57
 𝑌(𝑡) 𝑡≥0
◦ 𝑌(𝑡) = 𝑁 𝑡 , P 𝑡 , Ф 𝑡
◦ N 𝑡 = N1 𝑡 , N2 𝑡 , … , N 𝑛 𝑡
T
◦ 𝑃(𝑡) = P1 𝑡 , P2 𝑡 , … , Pn 𝑡
T
 Pi 𝑡 = 𝑝𝑖1 𝑡 , 𝑝𝑖2 𝑡 , … , 𝑝𝑖𝑟 𝑡
T
◦ Ф 𝑡 = Ф1 𝑡 , Ф2 𝑡 , … , Ф 𝑛 𝑡
T
 Фi = (𝑗, 𝛼𝑙) 𝑐𝑒𝑙𝑙𝑗 ∈ 𝑁𝑖 𝑎𝑛𝑑 𝑎𝑐𝑡𝑖𝑜𝑛 𝛼𝑙 has been chosen by 𝐿𝐴𝑗
 𝑑𝑖𝑗
𝛽
𝑌 𝑡
 the reward probability of action αj of LAi.
23/60
αj
𝑝𝑖1 𝑡
LAi
𝑝𝑖𝑗 𝑡
𝑝𝑖𝑟 𝑡
16/60

32. 252 /57
 𝑌(𝑡) 𝑡≥0
◦ 𝑌(𝑡) = 𝑁 𝑡 , P 𝑡 , Ф 𝑡
◦ N 𝑡 = N1 𝑡 , N2 𝑡 , … , N 𝑛 𝑡
T
◦ 𝑃(𝑡) = P1 𝑡 , P2 𝑡 , … , Pn 𝑡
T
 Pi 𝑡 = 𝑝𝑖1 𝑡 , 𝑝𝑖2 𝑡 , … , 𝑝𝑖𝑟 𝑡
T
◦ Ф 𝑡 = Ф1 𝑡 , Ф2 𝑡 , … , Ф 𝑛 𝑡
T
 𝑑𝑖𝑗
𝛽
𝑌 𝑡
 the reward probability of action αj of LAi.
 Time varying!!!
23/60
αj
𝑝𝑖1 𝑡
LAi
𝑝𝑖𝑗 𝑡
𝑝𝑖𝑟 𝑡
Process in
local
environment
Reward/
punishment
16/60

33. 252 /57
conditions
On restructuring signals
Proposition 1
conditions
On the environment
Of LAs
Analyzing the Entropy
and restructuring
tendency
17/60
Restructuring
tendency
 Structure updating rule is tendency decreasing
◦ There is an iteration 𝑡′
< 𝑡 which N 𝑡 = N∗
t

34. 252 /57
conditions
On restructuring
signals
Proposition 1
conditions
On the environment
Of LAs
Analyzing the Entropy
and restructuring
tendency
 There is function 𝑓𝑖𝑗
𝛽
(Pi) = 𝑑𝑖𝑗
𝛽
(𝑌(t))
◦ 𝑓𝑖𝑗
𝛽
(Pi)
 continuous
 monotonically decreasing function of 𝑝𝑖𝑗
◦ a change in 𝑝𝑖𝑗 affects
 primarily 𝑓𝑖𝑗
𝛽
(Pi)
 much lesser extent 𝑓𝑖𝑘
𝛽
(Pi) (j ≠ k)
24/6017/60

35. 252 /57
conditions
On restructuring
signals
Proposition 1
conditions
On the environment
Of LAs
Analyzing the Entropy
and restructuring
tendency
24/60
LA of each cell operates
in a nonstationary
environment
Lemma 3-1
lim
𝑡→∞
P(𝑡) = P∗
Theorem 3-1
Analyzing the
expediency
 Each LA attempts to equalize the penalty rates from the
actions.
17/60

36. 252 /57
conditions
On restructuring
signals
Proposition 1
conditions
On the environment
Of LAs
Analyzing the Entropy
and restructuring
tendency
24/60
LA of each cell operates
in a nonstationary
environment
Lemma 3-1
lim
𝑡→∞
P(𝑡) = P∗
Theorem 3-1
Analyzing the
expediency
 In the CLA we have the following
 𝑝𝑖1
∗
× 1 − 𝑓𝑖1
𝛽
(Pi
∗
) = 𝑝𝑖2
∗
× 1 − 𝑓𝑖2
𝛽
(Pi
∗
) =…= 𝑝𝑖𝑟
∗
× 1 − 𝑓𝑖𝑟
𝛽
(Pi
∗
)
17/60

37. 252 /57
conditions
On restructuring
signals
Proposition 1
conditions
On the environment
Of LAs
Analyzing the Entropy
and restructuring
tendency
24/60
LA of each cell operates
in a nonstationary
environment
Lemma 3-1
lim
𝑡→∞
P(𝑡) = P∗
Theorem 3-1
Analyzing the
expediency
Lemma 3-2
Theorem 3-2
 If (𝑝𝑖𝑗
∗
×(1- 𝑓𝑖𝑗
𝛽
(Pi
∗
))<
1
𝑟
−
σ 𝑗=1
𝑟
𝑓𝑖𝑗
𝛽
(Pi
pc
)
𝑟2 )
 then
 lim
𝑡→∞
𝐸 σ 𝑗 𝑝𝑖𝑗 𝑡 × 𝑓𝑖𝑗
𝛽
(Pi 𝑡 ) > lim
𝑡→∞
σ 𝑗 𝑝𝑖𝑗
𝑝𝑐
𝑡 × 𝑓𝑖𝑗
𝛽
Pi
pc
𝑡
CLA is
expedient
17/60

38. 252 /57
conditions
On restructuring
signals
Proposition 1
conditions
On the environment
Of LAs
Analyzing the Entropy
and restructuring
tendency
 lim
𝑡→∞
𝑇(𝑡) = 𝟎
◦ The restructuring tendency of the CLA approaches to 𝟎
 The cellular structure approaches to a fixed structure
 lim
𝑡→∞
𝐻(𝑡) = − σ 𝑘=1
𝑛 σ𝑙=1
𝑟 𝑘
𝑝 𝑘𝑙
∗
× ln 𝑝 𝑘𝑙
∗
= ℎ∗
◦ The entropy of the CLA approaches to ℎ∗
24/6017/60

39. 252 /57
conditions
On restructuring
signals
Proposition 1
conditions
On the environment
Of LAs
Analyzing the
Entropy and
restructuring
tendency
 Structure updating rule is tendency decreasing
◦ there is an iteration 𝑡′ < 𝑡 which N 𝑡 = N∗.
25/60
Restructuring
tendency
18/60

40. 252 /57
conditions
On restructuring
signals
conditions
On the environment
Of LAs
Analyzing the
Entropy and
restructuring
tendency
 There is function 𝑓𝑖𝑗
𝛽
(Pi) = 𝑑𝑖𝑗
𝛽
(𝑆(t))
◦ 𝑓𝑖𝑗
𝛽
(Pi)
 Continuous
 monotonically decreasing function of 𝑝𝑖𝑗
◦ a change in 𝑝𝑖𝑗 affects
 primarily 𝑓𝑖𝑗
𝛽
(Pi)
 much lesser extent 𝑓𝑖𝑘
𝛽
(Pi) (j ≠ k)
◦ There is an action 𝛼𝑙 which 𝑓𝑖𝑙
𝛽
(Pi) > 𝑓𝑖𝑘
𝛽
(Pi) where 𝑙 ≠ 𝑘.
25/6018/60

41. 252 /57
conditions
On restructuring
signals
conditions
On the environment
Of LAs
Analyzing the
Entropy and
restructuring
tendency
 The CADCLA can be ε-optimal with respect to cells
(Theorem 3-3)
◦ The CADCLA can be expedient with respect to cells (Theorem
3-4)
25/6018/60

42. 252 /57
conditions
On restructuring
signals
conditions
On the environment
Of LAs
Analyzing the
Entropy and
restructuring
tendency
 lim
𝑡→∞
inf 𝑝𝑖𝑙 𝑡 > 1 − 𝜀 (Theorem 3-3)
25/6018/60

43. 252 /57
conditions
On restructuring
signals
conditions
On the environment
Of LAs
Analyzing the
Entropy and
restructuring
tendency
 lim
𝑡→∞
𝑇(𝑡) = 𝟎
 lim
𝑡→∞
𝐻(𝑡) = ℎε
◦ If (ε → 0) then ℎε → 0
◦ The set of actions of the cells approaches to a fixed set
25/6018/60

44. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
◦ CADCLA
 Definition
 Behavior
 Applications in cognitive networks
 Topology mismatch problem
 Super-peer selection problem
◦ CADCLA-VL
 Other Works
 Conclusion
1

45. 252 /57
 Cognitive peer-to-peer network
◦ Cognitive engine based on CADCLA
 Cognitive engine for topology mismatch problem
19/60

46. 252 /57
 High communication delay
 High traffic
Underlay topology
Peer-to-peer
network
20/60

47. 252 /57
Topology
matching
algorithms
1. Algorithms that use
information about the location
of peers[46][47][48]
2. Algorithms that use
information about landmark
peers[10],[49]-[56]
1. Tree based algorithm[64]-
[66]
2. Heuristic algorithms (X-Bot and
PROP-O) [42], [43], [57], [59], [67]
3. Algorithms that use
information about local of peers
21/60

48. 252 /57
 Two phase
◦ Local search
◦ Exchange operation
 Problems
◦ Lack of adaptation
 neighborhood radius
 Small or large?
 exchange operation
 Conditions to start or finish?
22/60

49. 252 /57
 A CADCLA isomorphic to the peer-to-peer network is created.
 To manage the neighborhood radius of local search
◦ Each LA has two actions “Increase parameter " and " Decrease parameter ".
◦ The local rule is tuned
 To manage the exchange operator
◦ The structure updating rule is tuned
23/60
Cognitive Peer-to-Peer Network
celli cellj
cellk
celllcellm
celln
cello

50. 252 /57
 Structure updating rule.
◦ The rule is inspired from Schelling model
 Schelling segregation model
◦ If the portion of its neighbors which have similar attribute with
it is lower than a parameter z, the agent is unhappy and prefers
to change its neighbors in order to increase the number similar
neighbors.
◦
24/60

51. 252 /57
 Local rule
◦ set 𝜁𝑖 to 1 if the position of the cell is not appropriate and 0 otherwise.
◦ set β𝑖 to 1 if
 the position of the cell is appropriate and the action of the LA of celli is equal to
“decrease radious”.
 the position of the cell is inappropriate and the action of the LA of celli and the
majority of actions of immediate neighboring LAs of celli are equal to”
Increase parameter”
 Structure updating rule
◦ Change the neighborhood of the cell using swap operator
25/60

52. 252 /57
𝑝2
𝑝1
𝑝4
𝑝6
𝑝5
𝑝7
𝑝9
𝑝8
𝑝12𝑝3
𝑝11
𝑝13
𝑝10
26/60

53. 252 /57
 Simulation setup
◦ Oversim simulator
◦ Number of nodes is 10876
 Performance metrics
◦ OCD
 Sum of end-to-end delays of links
◦ CMO
 The number of control messages
27/60

54. 252 /5728/60
 Proposed cognitive engine performs better than PROP-
O and X-BOT algorithms
◦ parameter adaptation (neighborhood radius)
0
2000000
4000000
6000000
8000000
10000000
12000000
14000000
16000000
18000000
10 20 30 40 50 60 70 80 90 100
OCD(ms)
Round
PROP-OL
PROP-O
X-BOT

55. 252 /5729/60
 As the time passes the performance of the proposed
cognitive improves
◦ The peer is able to decrease its neighborhood radius.
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
10 20 30 40 50 60 70 80 90 100
CMO
Round
PROP-OL
PROP-O
X-BOT

56. 252 /57
 Mismatch problem
◦ Another version of the proposed cognitive engine
22/4030/60
Designing
PROP-OX
• CADCLA with
LRP learning
algorithm
Checking
conditions
• The reward
probabilities
are
decreasing
Evaluating the
results
• Entropy,
• Restructuring
tendency
• Average
reward

57. 252 /57
 Satisfying all conditions is not possible
◦ distributed and dynamic nature of the network!
 finding a function for computing reward probability is not possible
22/4031/60
Designing
PROP-OX
• CADCLA with
LRP learning
algorithm
Checking
conditions
• The reward
probabilities
are
decreasing
Evaluating the
results
• Entropy,
• Restructuring
tendency
• Average
reward

58. 252 /57
 Expediency was checked using average reward
metric
22/4032/60
Designing
PROP-OX
• CADCLA with
LRP learning
algorithm
Checking
conditions
• The reward
probabilities
are
decreasing
Evaluating the
results
• Entropy,
• Restructuring
tendency
• Average
reward

59. 252 /57
 Entropy is decreasing
22/4033/60
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
10 20 30 40 50 60 70 80 90 100
Entropy
Rounds

60. 252 /57
 Restructuring tendency is decreasing
22/4034/60
0
1000
2000
3000
4000
5000
6000
7000
8000
10 20 30 40 50 60 70 80 90 100
Restructuringtendency
Round

61. 252 /57
 Expediency check
◦ Average reward
22/4035/60
0
1000
2000
3000
4000
5000
6000
10 20 30 40 50 60 70 80 90 100
AvarageReward
Rounds
PROP-OP PROP-OX
Ali Mohammad Saghiri and Mohammad Reza Meybodi , " An Approach for Designing Cognitive Engines in Cognitive
Peer-to-Peer Networks", Journal of Network and Computer Applications, Vol. 70, pp. 17-40, 2016, DOI:
10.1016/j.jnca.2016.05.012

62. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
◦ CADCLA
 Definition
 Norm of behavior
 Applications in cognitive networks
 Topology mismatch problem
 Super-peer selection problem
◦ CADCLA-VL
 Other Works
 Conclusion
1

63. 252 /57
Super-peer
selection
1. Non-Adaptive [68]-[73]
2. Adaptive [35], [74]–[81]
1. Considering delays among
peer
2. Considering capacity of the
peer (Myconet, SG-1, and SPS
SG-LA) [35],[74],[77], [81]
36/60

64. 252 /57
 Structure updating rule.
◦ The rule is inspired from fungal growth pattern
 Fungal growth pattern
◦ Cells reorganize themselves considering the resources
◦ Some cells manages other cells
37/60

65. 252 /57
 An CADCLA isomorphic to the peer-to-peer network is
created.
◦ The attribute of celli consists of (capacity ci and type ti)
 The state machine is used to set the type of the cell.
◦ Each cell is equipped with a LA two actions
 Colony-Extender
 Colony- Immobilize.
24/4038/60
Ordinary peer
Super peer

66. 252 /57
 Structure updating rule uses Absorb operation
 the structure of CLA changes
◦ and as a results the configuration of the super-peer network
improves
25/4039/60
Ali Mohammad Saghiri and Mohammad Reza Meybodi , " An Adaptive Super-Peer Selection Algorithm Considering Peers
Capacity Utilizing Asynchronous Dynamic Cellular Learning Automata", Applied Intelligence, 2017, DOI:
10.117/s10489-017-0946-9

67. 252 /57
 Simulation setup
◦ Oversim simulator
◦ Number of nodes is 10000
 Performance metrics
◦ Entropy
◦ Restructuring Tendency
◦ NSP
 the number of super-peers
◦ CMO
 the number of control messages
26/4040/60

68. 252 /5727/4041/60
0
1000
2000
3000
4000
5000
6000
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
280
290
300
310
320
330
340
350
360
370
380
390
400
410
420
430
440
450
460
470
480
490
500
Entropy
Round

69. 252 /5727/4042/60
0
1000
2000
3000
4000
5000
6000
7000
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
280
290
300
310
320
330
340
350
360
370
380
390
400
410
420
430
440
450
460
470
480
490
500
restructuringtendency
Round

70. 252 /5743/60
0
2000
4000
6000
8000
10000
10 20 30 40 50 60 70 80 90 100
NSP
Round
NetworkSize=10000
SG-1 SG-LA SPS Myconet X-NET

71. 252 /5744/60
0
5000
10000
15000
20000
25000
10 20 30 40 50 60 70 80 90 100
CMO
Round
NetworkSize=10000
SG-1 SG-LA SPS Myconet X-NET

72. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
◦ CADCLA
◦ CADCLA-VL
 Definition
 Application
 Other Works
 Conclusion
11/40

73. 252 /57
 CADCLA with varying number of LAs in each cell (CADCLA-VL)
◦ Suitable when
 The number of LAs of the cells change over time
◦ Application for landmark clustering
28/4045/60
iLA
𝛹𝑖
𝛽𝑖 Action
iCell
Local Environment
Structure Updating Rule
𝑁𝑖
1
Ф𝑖, 𝑁𝑖
𝜁𝑖
Local Rule
𝛷𝑖 − 𝛷𝑖
1

74. 252 /57
Topology
matching
algorithms
1. Algorithms that use
information about the location
of peers[46][47][48]
2. Algorithms that use
information about landmark
peers(mOverlay and lOverlay)
[10],[49]-[56]
3. Algorithms that use
information about local of peers
[42], [43], [57], [59], [66],[67]
46/60

75. 252 /57
 Structure updating rule.
◦ The rule is inspired from Voronoi diagrams construction algorithm
 Voronoi Diagram
◦ In a Euclidean plane, the Voronoi diagram of a set of points is a
collection of cells that divide up the plane.
◦
47/60

76. 252 /57
 A CLA isomorphic to the
overlay network is
created.
 The LAs in each cell has
two actions
◦ "set the label to Ordinary
peer "
◦ "set the label to Landmark
peer".
peern
peeri
peerm
peero
peerj
peerp
peerk
peerq
peerr
peerl
Clusterk
Clusteri
Clusterj
Peer-to-peer
Overlay
Network
Underlay
Network
LAn LAi
LAm
LAo
LAj
LAp
LAk
LAq
LAr
LAl
Cellz
Cellx
Celly
CADCLA-VL
Voronoi
Diagram
29/4048/60
Ali Mohammad Saghiri and Mohammad Reza Meybodi, " A Closed Asynchronous Dynamic Model of Cellular Learning
Automata and its Application to Peer-to-Peer Networks", Genetic Programming and Evolvable Machines, 2017, DOI:
10.1007/s10710-017-9299-7

77. 252 /57
 Number of nodes 10000
 Performance metrics
◦ Entropy
◦ Restructuring Tendency
◦ TCD
 the total of all-pairs end-to-end communication delay.
◦ CMO
 the number of control messages
49/60

78. 252 /5750/60
0
1000
2000
3000
4000
5000
6000
7000
8000
10 20 30 40 50 60 70 80 90 100
Entropy
Round
xOverlay
kOverlay
nOverlay

79. 252 /5751/60
0
1000
2000
3000
4000
5000
6000
7000
10 20 30 40 50 60 70 80 90 100
RestructuringTendency
Round
xOverlay

80. 252 /5752/60
0
2000000
4000000
6000000
8000000
10000000
12000000
14000000
10000 20000 30000 40000 50000
TCD
network size
xOverlay
mOverlay
lOverlay

81. 252 /5753/60
0
200000
400000
600000
800000
1000000
1200000
1400000
10000 20000 30000 40000 50000
CMO
Network size
xOverlay
mOverlay
lOverlay

82. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
 Other Works
 Conclusion
11/40

83. 252 /57
 DCLAs
◦ OADCLA-VL[Open]
 Needs centralized computation
 Application in graph theory
 LA based Hybrid algorithms for topology mismatch
problem
◦ A heuristic algorithm based on LA and Schelling model
◦ A landmark clustering algorithm based on LA and mOverly
algorithm
54/60

84. 252 /57
 Overview
 Background
 Statement of the problem
 Dynamic models of CLAs
 Other Works
 Conclusion
◦ Contributions
◦ Future directions
◦ Publication
11/40

85. 252 /57
Closed Asynchronous DCLA (CADCLA), CADCLA-VL, OADCLA-VL
Cellular Learning Automata Theory
• An approach for designing Cognitive peer-to-peer networks
based on DCLA
• DCLA based algorithm for super-peer selection problem
• DCLA based algorithm for landmark clustering
• Two Learning Automata-based Algorithm for topology matching
in peer-to-peer networks
Peer-to-Peer Networks
OADCLA-VL based algorithm for allocation hub location
problem
Graph theory
Suggesting the conditions under which the CADCLA is expedient(LRP
and LRƐP)
Expediency Analysis
55/60

86. 252 /57
 Cognitive peer-to-peer networks based on DCLAs
◦ DCLA based self-organized mechanism
 Schelling model
 Fungal growth
 More attempts for extracting rules for the DCLA
 Analysis of the behaviors of the proposed models can
be pursued
◦ LRI and LPI learning algorithms
◦ LA with continuous action-set
56/60

87. 252
•Ali Mohammad Saghiri and Mohammad Reza Meybodi , " An Approach for
Designing Cognitive Engines in Cognitive Peer-to-Peer Networks", Journal of
Network and Computer Applications, Vol. 70, pp. 17-40, 2016, DOI:
10.1016/j.jnca.2016.05.012 [IF=3.5]
•Ali Mohammad Saghiri and Mohammad Reza Meybodi , " On Expediency of
Asynchronous Dynamic Cellular Learning Automata ", Journal of Computational
Science, 2017 [IF=1.7]
•Ali Mohammad Saghiri and Mohammad Reza Meybodi , " An Adaptive Super-
Peer Selection Algorithm Considering Peers Capacity Utilizing Asynchronous
Dynamic Cellular Learning Automata", Applied Intelligence, 2017, DOI:
10.117/s10489-017-0946-9 [IF=1.9]
CADCLA
•Ali Mohammad Saghiri and Mohammad Reza Meybodi, " A Closed Asynchronous
Dynamic Model of Cellular Learning Automata and its Application to Peer-to-Peer
Networks", Genetic Programming and Evolvable Machines, 2017, DOI:
10.1007/s10710-017-9299-7[IF=1.5]
CADCLA-
VL
•Ali Mohammad Saghiri and Mohammad Reza Meybodi , " Open
Dynamic Cellular Learning Automata and its Application to Allocation
Hub Location Problem", Knowledge Based System[submitted]
OADCLA-
VL
57/60

88. 252
•Ali Mohammad Saghiri and Mohammad Reza Meybodi , " A Distributed
Adaptive Landmark Clustering Algorithm Based on mOverlay and Learning
Automata for Topology Mismatch Problem in Unstructured Peer-to-Peer
Networks", International journal of communication systems, Vol. 30, No. 3, pp.
1-22, 2017, DOI: 10.1002/dac.297[IF=1]
•Ali Mohammad Saghiri and Mohammad Reza Meybodi , " A Self-adaptive
Algorithm for Topology Matching in Unstructured Peer-to-Peer Networks",
Journal of network and systems management, Vol. 24, pp. 393-426, 2016,
DOI: 10.1007/s10922-015-9353-9 [IF=1.5]
•Sara Fathipour, Ali Mohammad Saghiri and Mohammad Reza Meybodi , " A
Delay Aware Super-Peer Selection Algorithm for Gradient Topology Utilizing
Learning Automata", Wireless Personal Communications, 2017, DOI:
10.1007/s11277-017-3943-7 [IF=0.9]
•Nahid AmirAzodi, Ali Mohammad Saghiri and Mohammad Reza Meybodi , " An
Adaptive Algorithm for Super-Peer Selection Considering Peer's Capacity in
Mobile Peer-to-Peer Networks based on Learning Automata", Peer-to-Peer
Networking and Applications, 2016, DOI: 10.1007/s12083-016-0503-y
[IF=1.2]
LA based
algorithms
58/60

89. Thank You for
your attention
Thank You