Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
×

# Big Data Analysis with Signal Processing on Graphs

1,412 views

Published on

Talk no. 8 about big data given by Mohamed Seif

Published in: Engineering
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Sandryhaila, Aliaksei, and Jose MF Moura. "Big data analysis with signal processing on graphs: Representation and processing of massive data sets with irregular structure." Signal Processing Magazine, IEEE 31.5 (2014): 80-90.

Are you sure you want to  Yes  No
Your message goes here
• Errata in slide no. 7: E= {{1,2},{2,4},{3,4},{3,1}}

Are you sure you want to  Yes  No
Your message goes here

### Big Data Analysis with Signal Processing on Graphs

1. 1. Big Data Talks Nile University Talk 8 Big Data Analysis with Signal Processing on Graphs Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Mohamed Seif m.seif@nu.edu.eg 8-1
2. 2. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-2
3. 3. Introduction How to relate the Signal Processing Deﬁnitions with Graphs? Big Data Analysis with Signal Processing on Graphs 8-3
4. 4. Introduction How to relate the Signal Processing Deﬁnitions with Graphs? Big Data Analysis with Signal Processing on Graphs 8-4
5. 5. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-5
6. 6. What Is Graph? In graph theory, the graph G is deﬁned as a tuple G = (V, E) where V = {v0, v1, . . . , vN−1} is the set of N nodes and E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links between the nodes. Example: V = {1, 2, 3, 4} E = {{1, 2}, {2, 3}, {3, 4}, {4, 1}} 1 2 3 4 Big Data Analysis with Signal Processing on Graphs 8-6
7. 7. What Is Graph? In graph theory, the graph G is deﬁned as a tuple G = (V, E) where V = {v0, v1, . . . , vN−1} is the set of N nodes and E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links between the nodes. Example: V = {1, 2, 3, 4} E = {{1, 2}, {2, 3}, {3, 4}, {4, 1}} 1 2 3 4 Big Data Analysis with Signal Processing on Graphs 8-7
8. 8. What Is Graph? (cont’d) In graph theory, the graph G is deﬁned as a tuple G = (V, E) where V = {v0, v1, . . . , vN−1} is the set of N nodes and E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links between the nodes. Big Data Analysis with Signal Processing on Graphs 8-8
9. 9. What Is Graph? In graph theory, the graph G is deﬁned as a tuple G = (V, E) where V = {v0, v1, . . . , vN−1} is the set of N nodes and E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links between the nodes. Big Data Analysis with Signal Processing on Graphs 8-9
10. 10. Alternative Representation of Graphs 1. Adjacency matrix AN×N , deﬁned as Ai,j =    1 if vi & vj are connected 0 o.w. 2. Laplacian graph LN×N , deﬁned as L = D − A, where D is the degree matrix Li,j =    deg(vi) if i = j −1 if i = j & vi is adjacent to vj 0 o.w. Big Data Analysis with Signal Processing on Graphs 8-10
11. 11. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-11
12. 12. Graph Signals Given the graph, the data set forms a graph signal, deﬁned as a map s : V → C, vn → sn (1) It is convenient to write graph signals as vectors s = [s0, s1, . . . , sN−1] T ∈ CN×1 (2) Big Data Analysis with Signal Processing on Graphs 8-12
13. 13. Graph Shift In DSP, a signal shift, implemented as a time delay ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = Cs (3) where C is the N × N cyclic shift matrix. DSP on Graphs extends the concept of shift to general graphs by deﬁning the graph shift as a local operation that replaces a signal value sn at node vn by a linear combination of the values at neighbors of vn weighted by their edge weights: ˜sn = m∈Nn An,msm (4) Big Data Analysis with Signal Processing on Graphs 8-13
14. 14. Graph Shift (cont’d) It can be interpreted as a ﬁrst-order interpolation, weighted averaging, or regression on graphs, which is a widely used operation in graph regression, distributed consensus, telecommunications. Then, the graph shift is written as ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = As (5) Big Data Analysis with Signal Processing on Graphs 8-14
15. 15. Graph Filters and Z-Transform In signal processing, a ﬁlter is a system H(.) that takes an input signal s and outputs a signal: ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = H(s) (6) Among the most widely used ﬁlters are linear shift-ivariant (LSI) ones. The z-transform provides a convenient representation for signals and ﬁlters in DSP. (In short) An alternative representation for the output signal is given by ˜s = h(C)s (7) where h(c) = N−1 n=0 hnCn (Resultant is a circulant matrix) Big Data Analysis with Signal Processing on Graphs 8-15
16. 16. Graph Filters and Z-Transform In signal processing, a ﬁlter is a system H(.) that takes an input signal s and outputs a signal: ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = H(s) (6) Among the most widely used ﬁlters are linear shift-ivariant (LSI) ones. The z-transform provides a convenient representation for signals and ﬁlters in DSP. (In short) An alternative representation for the output signal is given by ˜s = h(C)s (7) where h(c) = N−1 n=0 hnCn (Resultant is a circulant matrix) Big Data Analysis with Signal Processing on Graphs 8-16
17. 17. Graph Filters and Z-Transform In signal processing, a ﬁlter is a system H(.) that takes an input signal s and outputs a signal: ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = H(s) (6) Among the most widely used ﬁlters are linear shift-ivariant (LSI) ones. The z-transform provides a convenient representation for signals and ﬁlters in DSP. (In short) An alternative representation for the output signal is given by ˜s = h(C)s (7) where h(c) = N−1 n=0 hnCn (Resultant is a circulant matrix) Big Data Analysis with Signal Processing on Graphs 8-17
18. 18. Graph Filters and Z-Transform In signal processing, a ﬁlter is a system H(.) that takes an input signal s and outputs a signal: ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = H(s) (6) Among the most widely used ﬁlters are linear shift-ivariant (LSI) ones. The z-transform provides a convenient representation for signals and ﬁlters in DSP. (In short) An alternative representation for the output signal is given by ˜s = h(C)s (7) where h(c) = N−1 n=0 hnCn (Resultant is a circulant matrix) Big Data Analysis with Signal Processing on Graphs 8-18
19. 19. Graph Filters and Z-Transform (cont’d) DSP on Graphs extends the concept of ﬁlters to general graphs. Similarly to the extension of the time shift to the graph shift, ﬁlters are generalized to graph ﬁlters as polynomials in the graph shift , and all LSI graph ﬁlters have the form h(A) = L−1 l=0 hlAl (8) In analogy with signal ﬁlters, the graph ﬁlter output is given by ˜s = h(A)s (9) Big Data Analysis with Signal Processing on Graphs 8-19
20. 20. Graph Fourier Transform Mathematically, a Fourier transform with respect to a set of operators is the expansion of a signal into a basis of the operators eigen functions. Since in signal processing the operators of interest are ﬁlters, DSPG deﬁnes the Fourier transform with respect to the graph ﬁlters. ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = GFT{s} (10) Big Data Analysis with Signal Processing on Graphs 8-20
21. 21. Graph Fourier Transform Mathematically, a Fourier transform with respect to a set of operators is the expansion of a signal into a basis of the operators eigen functions. Since in signal processing the operators of interest are ﬁlters, DSPG deﬁnes the Fourier transform with respect to the graph ﬁlters. ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = GFT{s} (10) Big Data Analysis with Signal Processing on Graphs 8-21
22. 22. Graph Fourier Transform (cont’d) For simplicity, assume that A is diagonalizable and its decomposition is A = V ΛV −1 (11) where the columns vn of the matrix V = [v0 · · · vN−1] ∈ CN×N are the eigenvectors of A and Λ = diag(λ0, . . . , λN−1) are eigenvalues of A In general A can be diagonalized using Jordan decomposition. Big Data Analysis with Signal Processing on Graphs 8-22
23. 23. Graph Fourier Transform (cont’d) For simplicity, assume that A is diagonalizable and its decomposition is A = V ΛV −1 (11) where the columns vn of the matrix V = [v0 · · · vN−1] ∈ CN×N are the eigenvectors of A and Λ = diag(λ0, . . . , λN−1) are eigenvalues of A In general A can be diagonalized using Jordan decomposition. Big Data Analysis with Signal Processing on Graphs 8-23
24. 24. Graph Fourier Transform (cont’d) The eigenfunctions of graph ﬁlters h(A) are given by the eigenvectors of the graph shift matrix A Since the expansion into the eigenbasis is given by the multiplication with the inverse eigenvector matrix, which always exists, the graph Fourier transform is well deﬁned and computed as ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1] T = V −1 s (12) = Fs (13) where F = V −1 is the graph Fourier transform matrix. Big Data Analysis with Signal Processing on Graphs 8-24
25. 25. Graph Fourier Transform (cont’d) The eigenfunctions of graph ﬁlters h(A) are given by the eigenvectors of the graph shift matrix A Since the expansion into the eigenbasis is given by the multiplication with the inverse eigenvector matrix, which always exists, the graph Fourier transform is well deﬁned and computed as ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1] T = V −1 s (12) = Fs (13) where F = V −1 is the graph Fourier transform matrix. Big Data Analysis with Signal Processing on Graphs 8-25
26. 26. Graph Fourier Transform (cont’d) The eigenfunctions of graph ﬁlters h(A) are given by the eigenvectors of the graph shift matrix A Since the expansion into the eigenbasis is given by the multiplication with the inverse eigenvector matrix, which always exists, the graph Fourier transform is well deﬁned and computed as ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1] T = V −1 s (12) = Fs (13) where F = V −1 is the graph Fourier transform matrix. Big Data Analysis with Signal Processing on Graphs 8-26
27. 27. Graph Fourier Transform (cont’d) The inverse graph Fourier transform reconstructs the graph signal from is frequency content by combining graph frequency components weighted by the coeﬃcients of the signal’s graph Fourier transform: s = ˆs0v0 + ˆs1v1 + · · · + ˆsN−1vN−1 (14) = F−1 s = V s (15) Big Data Analysis with Signal Processing on Graphs 8-27
28. 28. Low and High Frequencies on Graphs The values ˆsn in (12) are the signal’s expansion in the eigenvector basis and represent the frequency content of the signal s. The eigenvalues λn of the shift matrix A represent graph frequency content, and the eigenvectors vn represent the corresponding graph frequency component. To conclude, the higher λn, the higher frequency content and vice versa. Big Data Analysis with Signal Processing on Graphs 8-28
29. 29. Low and High Frequencies on Graphs The values ˆsn in (12) are the signal’s expansion in the eigenvector basis and represent the frequency content of the signal s. The eigenvalues λn of the shift matrix A represent graph frequency content, and the eigenvectors vn represent the corresponding graph frequency component. To conclude, the higher λn, the higher frequency content and vice versa. Big Data Analysis with Signal Processing on Graphs 8-29
30. 30. Low and High Frequencies on Graphs The values ˆsn in (12) are the signal’s expansion in the eigenvector basis and represent the frequency content of the signal s. The eigenvalues λn of the shift matrix A represent graph frequency content, and the eigenvectors vn represent the corresponding graph frequency component. To conclude, the higher λn, the higher frequency content and vice versa. Big Data Analysis with Signal Processing on Graphs 8-30
31. 31. Low and High Frequencies on Graphs The values ˆsn in (12) are the signal’s expansion in the eigenvector basis and represent the frequency content of the signal s. The eigenvalues λn of the shift matrix A represent graph frequency content, and the eigenvectors vn represent the corresponding graph frequency component. To conclude, the higher λn, the higher frequency content and vice versa. Big Data Analysis with Signal Processing on Graphs 8-31
32. 32. Frequency Response of Graph Filters In addition to expressing the frequency content of graph signals, the graph Fourier transform also characterizes the eﬀect of ﬁlters on the frequency content of signals. The ﬁltering operation ˜s = h(A)s can be written using h(A) = L−1 l=0 hlAl and ˆs = V −1 s as follows ˜s = h(A)s = h(F−1 AF)s = F−1 h(Λ)Fs (16) where h(Λ) is a diagonal matrix with values h(λn) = L−1 l=0 hlλl n As a result, ˜s = h(A)s ⇔ Fs = h(Λ)s (17) Big Data Analysis with Signal Processing on Graphs 8-32
33. 33. Frequency Response of Graph Filters In addition to expressing the frequency content of graph signals, the graph Fourier transform also characterizes the eﬀect of ﬁlters on the frequency content of signals. The ﬁltering operation ˜s = h(A)s can be written using h(A) = L−1 l=0 hlAl and ˆs = V −1 s as follows ˜s = h(A)s = h(F−1 AF)s = F−1 h(Λ)Fs (16) where h(Λ) is a diagonal matrix with values h(λn) = L−1 l=0 hlλl n As a result, ˜s = h(A)s ⇔ Fs = h(Λ)s (17) Big Data Analysis with Signal Processing on Graphs 8-33
34. 34. Frequency Response of Graph Filters In addition to expressing the frequency content of graph signals, the graph Fourier transform also characterizes the eﬀect of ﬁlters on the frequency content of signals. The ﬁltering operation ˜s = h(A)s can be written using h(A) = L−1 l=0 hlAl and ˆs = V −1 s as follows ˜s = h(A)s = h(F−1 AF)s = F−1 h(Λ)Fs (16) where h(Λ) is a diagonal matrix with values h(λn) = L−1 l=0 hlλl n As a result, ˜s = h(A)s ⇔ Fs = h(Λ)s (17) Big Data Analysis with Signal Processing on Graphs 8-34
35. 35. Frequency Response of Graph Filters (cont’d) That is, the frequency content of a ﬁltered signal is modiﬁed by multiplying its frequency content element-wise by h(λn) . These values represent the graph frequency response of the graph ﬁlter. The relation is a generalization of the classical convolution theorem to graphs: ﬁltering a graph signal in the graph domain is equivalent in the frequency domain to multiplying the signal spectrum by the frequency response of the graph ﬁlter. ˜s = h(A)s ⇔ Fs = h(Λ)s Big Data Analysis with Signal Processing on Graphs 8-35
36. 36. Frequency Response of Graph Filters (cont’d) That is, the frequency content of a ﬁltered signal is modiﬁed by multiplying its frequency content element-wise by h(λn) . These values represent the graph frequency response of the graph ﬁlter. The relation is a generalization of the classical convolution theorem to graphs: ﬁltering a graph signal in the graph domain is equivalent in the frequency domain to multiplying the signal spectrum by the frequency response of the graph ﬁlter. ˜s = h(A)s ⇔ Fs = h(Λ)s Big Data Analysis with Signal Processing on Graphs 8-36
37. 37. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-37
38. 38. Product Graphs Consider two graphs G1 = (V1, A1) and G2 = (V2, A2) with |V1| = N1 and |V2| = N2 nodes, respectively. The product graph, denoted by , of G1 and G2 is the graph G = G1 G2 = (V, A ) (18) with |V| = N1N2 and dim(A ) = N1N2 × N1N2. Big Data Analysis with Signal Processing on Graphs 8-38
39. 39. Common Product Graphs Types 1. Kronecker product A⊗ = A1 ⊗ A2 (19) Example: If we have two matrices B ∈ CM×N and C ∈ CK×L , then the Kronecker product is deﬁned as follows B ⊗ C =     b1,1C ... bM,1C · · · ... · · · b1,M C ... bM,M C     ∈ CMK×NL (20) Big Data Analysis with Signal Processing on Graphs 8-39
40. 40. Common Product Graphs Types 1. Cartesian product A× = A1 ⊗ IN2 + IN1 ⊗ A2 (21) 2. Strong product A = A1 ⊗ A2 + A1 ⊗ IN2 + IN1 ⊗ A2 (22) Big Data Analysis with Signal Processing on Graphs 8-40
41. 41. Examples on Product Graphs Big Data Analysis with Signal Processing on Graphs 8-41
42. 42. Examples on Product Graphs Big Data Analysis with Signal Processing on Graphs 8-42
43. 43. Examples on Product Graphs Big Data Analysis with Signal Processing on Graphs 8-43
44. 44. Notes on Product Graphs Big Data Analysis with Signal Processing on Graphs 8-44
45. 45. Signal Processing on Product Graphs The computation of ﬁltering and Fourier transform on graphs and improve algorithms, data storage, and memory access for large data sets can modularized thanks to graph products. Such as Filtering Computation complexity: O(N2 ) =⇒ O(N(N1 + N2)) Fourier transform Computation complexity: O(N3 ) =⇒ O(N3 1 + N3 2 ) Big Data Analysis with Signal Processing on Graphs 8-45
46. 46. Signal Processing on Product Graphs The computation of ﬁltering and Fourier transform on graphs and improve algorithms, data storage, and memory access for large data sets can modularized thanks to graph products. Such as Filtering Computation complexity: O(N2 ) =⇒ O(N(N1 + N2)) Fourier transform Computation complexity: O(N3 ) =⇒ O(N3 1 + N3 2 ) Big Data Analysis with Signal Processing on Graphs 8-46
47. 47. Signal Processing on Product Graphs The computation of ﬁltering and Fourier transform on graphs and improve algorithms, data storage, and memory access for large data sets can modularized thanks to graph products. Such as Filtering Computation complexity: O(N2 ) =⇒ O(N(N1 + N2)) Fourier transform Computation complexity: O(N3 ) =⇒ O(N3 1 + N3 2 ) Big Data Analysis with Signal Processing on Graphs 8-47
48. 48. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-48
49. 49. Applications Like-wise traditional DSP problems: Data compression Fourier transform or through wavelet expansions, or adaptive ﬁlter design Detection of corrupted data High pass ﬁlter Big Data Analysis with Signal Processing on Graphs 8-49
50. 50. Applications Like-wise traditional DSP problems: Data compression Fourier transform or through wavelet expansions, or adaptive ﬁlter design Detection of corrupted data High pass ﬁlter Big Data Analysis with Signal Processing on Graphs 8-50
51. 51. Applications Like-wise traditional DSP problems: Data compression Fourier transform or through wavelet expansions, or adaptive ﬁlter design Detection of corrupted data High pass ﬁlter Big Data Analysis with Signal Processing on Graphs 8-51
52. 52. Challenges of Big Data While there is no single, universally agreed upon set of properties that deﬁne big data. Some of the commonly mentioned ones are volume, velocity, and variety of data. Big Data Analysis with Signal Processing on Graphs 8-52
53. 53. Challenges of Big Data While there is no single, universally agreed upon set of properties that deﬁne big data. Some of the commonly mentioned ones are volume, velocity, and variety of data. Big Data Analysis with Signal Processing on Graphs 8-53
54. 54. Challenges of Big Data Some of the commonly mentioned ones are volume, velocity, and variety of data. First of all, the sheer volume of data to be processed requires eﬃcient distributed and scalable storage, access, and processing. High velocity of new data arrival demands fast algorithms to prevent bottlenecks and explosion of the data volume and to extract valuable information from the data and incorporate it into the decision-making process in real time. (FFT in DSP) Finally, collected data sets contain information in all varieties and forms, including numerical, textual, and visual data. To generalize data analysis techniques to diverse data sets, we need a common representation framework for data sets and their structure. Big Data Analysis with Signal Processing on Graphs 8-54
55. 55. Challenges of Big Data Some of the commonly mentioned ones are volume, velocity, and variety of data. First of all, the sheer volume of data to be processed requires eﬃcient distributed and scalable storage, access, and processing. High velocity of new data arrival demands fast algorithms to prevent bottlenecks and explosion of the data volume and to extract valuable information from the data and incorporate it into the decision-making process in real time. (FFT in DSP) Finally, collected data sets contain information in all varieties and forms, including numerical, textual, and visual data. To generalize data analysis techniques to diverse data sets, we need a common representation framework for data sets and their structure. Big Data Analysis with Signal Processing on Graphs 8-55
56. 56. Challenges of Big Data Some of the commonly mentioned ones are volume, velocity, and variety of data. First of all, the sheer volume of data to be processed requires eﬃcient distributed and scalable storage, access, and processing. High velocity of new data arrival demands fast algorithms to prevent bottlenecks and explosion of the data volume and to extract valuable information from the data and incorporate it into the decision-making process in real time. (FFT in DSP) Finally, collected data sets contain information in all varieties and forms, including numerical, textual, and visual data. To generalize data analysis techniques to diverse data sets, we need a common representation framework for data sets and their structure. Big Data Analysis with Signal Processing on Graphs 8-56
57. 57. Challenges of Big Data Some of the commonly mentioned ones are volume, velocity, and variety of data. First of all, the sheer volume of data to be processed requires eﬃcient distributed and scalable storage, access, and processing. High velocity of new data arrival demands fast algorithms to prevent bottlenecks and explosion of the data volume and to extract valuable information from the data and incorporate it into the decision-making process in real time. (FFT in DSP) Finally, collected data sets contain information in all varieties and forms, including numerical, textual, and visual data. To generalize data analysis techniques to diverse data sets, we need a common representation framework for data sets and their structure. Big Data Analysis with Signal Processing on Graphs 8-57
58. 58. Thank You! Big Data Analysis with Signal Processing on Graphs 8-58