1. Overlapping Community detection using Bayesian Non
Negative Matrix Factorization
Rajkumar Singh
Rishi Barua
Guide: Dr. Ashish Anand
Dept. of Computer Science
Indian Institute of Technology, Guwahati
April 18, 2013
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 1 / 15

2. Network Paradigm
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 2 / 15

3. Network Paradigm
V ∈ RN×N is the adjacency matrix
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 2 / 15

4. Network Paradigm
V ∈ RN×N is the adjacency matrix
vij can be boolean or denote connection weight
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 2 / 15

5. Network Paradigm
V ∈ RN×N is the adjacency matrix
vij can be boolean or denote connection weight
ki = j vij degree of node i
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 2 / 15

6. Network Paradigm
V ∈ RN×N is the adjacency matrix
vij can be boolean or denote connection weight
ki = j vij degree of node i
data in relational form
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 2 / 15

7. Commnity Detection
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 3 / 15

8. Commnity Detection
Q. What is Community ?
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 3 / 15

9. Commnity Detection
Q. What is Community ?
As such no Specific Definition. It is Context Dependent.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 3 / 15

10. Commnity Detection
Q. What is Community ?
As such no Specific Definition. It is Context Dependent.
Defn. : Here we define communities as Modules which are subgraphs with
more links connecting the nodes inside than outside them.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 3 / 15

11. Commnity Detection
Q. What is Community ?
As such no Specific Definition. It is Context Dependent.
Defn. : Here we define communities as Modules which are subgraphs with
more links connecting the nodes inside than outside them.
A given real world network is assumed to be clustered into a number
of latent classes of nodes.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 3 / 15

12. Commnity Detection
Q. What is Community ?
As such no Specific Definition. It is Context Dependent.
Defn. : Here we define communities as Modules which are subgraphs with
more links connecting the nodes inside than outside them.
A given real world network is assumed to be clustered into a number
of latent classes of nodes.
These nodes form regions of increased connectivity in the network.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 3 / 15

13. Commnity Detection
Q. What is Community ?
As such no Specific Definition. It is Context Dependent.
Defn. : Here we define communities as Modules which are subgraphs with
more links connecting the nodes inside than outside them.
A given real world network is assumed to be clustered into a number
of latent classes of nodes.
These nodes form regions of increased connectivity in the network.
These communities usually reflect functional modules that affect the
overall behavior of the system.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 3 / 15

14. Commnity Detection
Q. What is Community ?
As such no Specific Definition. It is Context Dependent.
Defn. : Here we define communities as Modules which are subgraphs with
more links connecting the nodes inside than outside them.
A given real world network is assumed to be clustered into a number
of latent classes of nodes.
These nodes form regions of increased connectivity in the network.
These communities usually reflect functional modules that affect the
overall behavior of the system.
Examples: Friend cliques in social networks, Similar proteins in a
protein interaction network, Research groups in a scientific
collaboration network
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 3 / 15

15. Community Detection
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 4 / 15

16. Non-negative Matrix Factorzation
Decompose data matrix V to a product of two other matrices W , H under
non-negativity constraints.
V ∼ ˆV = WH V , ˆV ∈ RF×N and W ∈ RF×K , H ∈ RK×N so that,
FK + KN < FN
Non-negativity constraints avoid the problem of an ill-posed solution
They also reflect the idea of parts-based representation. V can be
expressed as additive combination of certain basis structures defined
by wk, given an encoding hk
ˆV =
k
w:khk: (1)
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 5 / 15

17. Poisson Model
Given an Adjacency matrix V ∈ RN×N
Assume, pairwise interactions vij is generated by a Poisson
distribution with rate ˆvij .
Hence ˆV ∼ V
Expectation network ˆV is composed of W and H, so that ˆV = WH
Inner rank K : Unknown number of communities
wik, hkj ∈ 1, . . . , K: The contribution of each latent community to
ˆvij = k wikhkj
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 6 / 15

18. Cost Function
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 7 / 15

19. Cost Function
posterior:
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 7 / 15

20. Cost Function
posterior:
p(V |W , H, β) = p(V |W , H)p(W |β)p(H|β)p(β) (2)
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 7 / 15

21. Cost Function
posterior:
p(V |W , H, β) = p(V |W , H)p(W |β)p(H|β)p(β) (2)
by taking the − log of: p(V , W , H)p(W |β)p(H|β)p(β),
we define cost as U.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 7 / 15

22. Cost Function
posterior:
p(V |W , H, β) = p(V |W , H)p(W |β)p(H|β)p(β) (2)
by taking the − log of: p(V , W , H)p(W |β)p(H|β)p(β),
we define cost as U.
U = − log p(V |W , H) − log p(W |β) − log p(W |β) − log p(β) (3)
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 7 / 15

23. Cost Function
posterior:
p(V |W , H, β) = p(V |W , H)p(W |β)p(H|β)p(β) (2)
by taking the − log of: p(V , W , H)p(W |β)p(H|β)p(β),
we define cost as U.
U = − log p(V |W , H) − log p(W |β) − log p(W |β) − log p(β) (3)
U is our minimization objective under non-negativity constraints.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 7 / 15

24. Parameter inference
U = i j vij log
vij
ˆvij
+ ˆvij
+
1
2 k i βkw2
ik + j βkh2
kj − 2N log βk
+ k βkbk − (ak − 1) log βk + k
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 8 / 15

25. Parameter inference
U = i j vij log
vij
ˆvij
+ ˆvij
+
1
2 k i βkw2
ik + j βkh2
kj − 2N log βk
+ k βkbk − (ak − 1) log βk + k
We find w∗, h∗,β∗ that minimize U using the gradient descent
algorithm.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 8 / 15

26. Parameter inference
U = i j vij log
vij
ˆvij
+ ˆvij
+
1
2 k i βkw2
ik + j βkh2
kj − 2N log βk
+ k βkbk − (ak − 1) log βk + k
We find w∗, h∗,β∗ that minimize U using the gradient descent
algorithm.
The effective number of communities K is the number of non-zero
columns of W∗ and H∗
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 8 / 15

27. Results
W∗, H∗ describe a bipartite network of node allocations to
communities.
If our original adjacency matrix V is symmetric, then W∗ = HT
∗ .
Each wik or hki denotes the participation strength of node i to
community k.
The ith row of W or column of H describes a soft-membership
distribution of node i across communities.
Varying node participation scores allow us to describe overlaps
between communities in a disciplined manner.
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 9 / 15

28. Example: 1
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 10 / 15

29. Example: 1
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 11 / 15

30. Example: 1
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 12 / 15

31. Example: 2
We now take a 1000X1000 Adjacency matrix V , which is very sparse.
The graph is a non-community graph, i.e. having no community
structure.
Several of the nodes are isolated.
Solutions from EO, Louvian (as described in the paper) offer higher
modularity than NMF.
NMF clearly shows that the modularity is low and the nodes have no
preference to lie in any particular community, or are non-communal.
NMF does not suffer from resolution limit of modularity (which
groups smaller communities), as can be clearly seen.
The Output shows 1 community, with 428 isolated nodes i.e.
Non-communal
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 13 / 15

32. Example: 2
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 14 / 15

33. References
1 Overlapping community detection using Bayesian non-negative matrix
factorization - I. Psorakis, S. Roberts and M.Ebden
2 Signal Processing with Adaptive Sparse Structured Representations -
V.Tan and C.Fevotte
3 D.D. Lee and H.S. Seung, Nature ,401
R. Singh, R. Barua (IITG) Overlapping Community detection using Bayesian Non Negative Matrix FactorizationApril 18, 2013 15 / 15