Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Data Analysis with Signal Processing on Graphs

1,412 views

Published on

Talk no. 8 about big data given by Mohamed Seif

Published in: Engineering

Big Data Analysis with Signal Processing on Graphs

  1. 1. Big Data Talks Nile University Talk 8 Big Data Analysis with Signal Processing on Graphs Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Mohamed Seif m.seif@nu.edu.eg 8-1
  2. 2. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-2
  3. 3. Introduction How to relate the Signal Processing Definitions with Graphs? Big Data Analysis with Signal Processing on Graphs 8-3
  4. 4. Introduction How to relate the Signal Processing Definitions with Graphs? Big Data Analysis with Signal Processing on Graphs 8-4
  5. 5. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-5
  6. 6. What Is Graph? In graph theory, the graph G is defined as a tuple G = (V, E) where V = {v0, v1, . . . , vN−1} is the set of N nodes and E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links between the nodes. Example: V = {1, 2, 3, 4} E = {{1, 2}, {2, 3}, {3, 4}, {4, 1}} 1 2 3 4 Big Data Analysis with Signal Processing on Graphs 8-6
  7. 7. What Is Graph? In graph theory, the graph G is defined as a tuple G = (V, E) where V = {v0, v1, . . . , vN−1} is the set of N nodes and E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links between the nodes. Example: V = {1, 2, 3, 4} E = {{1, 2}, {2, 3}, {3, 4}, {4, 1}} 1 2 3 4 Big Data Analysis with Signal Processing on Graphs 8-7
  8. 8. What Is Graph? (cont’d) In graph theory, the graph G is defined as a tuple G = (V, E) where V = {v0, v1, . . . , vN−1} is the set of N nodes and E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links between the nodes. Big Data Analysis with Signal Processing on Graphs 8-8
  9. 9. What Is Graph? In graph theory, the graph G is defined as a tuple G = (V, E) where V = {v0, v1, . . . , vN−1} is the set of N nodes and E = {eij, ∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links between the nodes. Big Data Analysis with Signal Processing on Graphs 8-9
  10. 10. Alternative Representation of Graphs 1. Adjacency matrix AN×N , defined as Ai,j =    1 if vi & vj are connected 0 o.w. 2. Laplacian graph LN×N , defined as L = D − A, where D is the degree matrix Li,j =    deg(vi) if i = j −1 if i = j & vi is adjacent to vj 0 o.w. Big Data Analysis with Signal Processing on Graphs 8-10
  11. 11. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-11
  12. 12. Graph Signals Given the graph, the data set forms a graph signal, defined as a map s : V → C, vn → sn (1) It is convenient to write graph signals as vectors s = [s0, s1, . . . , sN−1] T ∈ CN×1 (2) Big Data Analysis with Signal Processing on Graphs 8-12
  13. 13. Graph Shift In DSP, a signal shift, implemented as a time delay ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = Cs (3) where C is the N × N cyclic shift matrix. DSP on Graphs extends the concept of shift to general graphs by defining the graph shift as a local operation that replaces a signal value sn at node vn by a linear combination of the values at neighbors of vn weighted by their edge weights: ˜sn = m∈Nn An,msm (4) Big Data Analysis with Signal Processing on Graphs 8-13
  14. 14. Graph Shift (cont’d) It can be interpreted as a first-order interpolation, weighted averaging, or regression on graphs, which is a widely used operation in graph regression, distributed consensus, telecommunications. Then, the graph shift is written as ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = As (5) Big Data Analysis with Signal Processing on Graphs 8-14
  15. 15. Graph Filters and Z-Transform In signal processing, a filter is a system H(.) that takes an input signal s and outputs a signal: ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = H(s) (6) Among the most widely used filters are linear shift-ivariant (LSI) ones. The z-transform provides a convenient representation for signals and filters in DSP. (In short) An alternative representation for the output signal is given by ˜s = h(C)s (7) where h(c) = N−1 n=0 hnCn (Resultant is a circulant matrix) Big Data Analysis with Signal Processing on Graphs 8-15
  16. 16. Graph Filters and Z-Transform In signal processing, a filter is a system H(.) that takes an input signal s and outputs a signal: ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = H(s) (6) Among the most widely used filters are linear shift-ivariant (LSI) ones. The z-transform provides a convenient representation for signals and filters in DSP. (In short) An alternative representation for the output signal is given by ˜s = h(C)s (7) where h(c) = N−1 n=0 hnCn (Resultant is a circulant matrix) Big Data Analysis with Signal Processing on Graphs 8-16
  17. 17. Graph Filters and Z-Transform In signal processing, a filter is a system H(.) that takes an input signal s and outputs a signal: ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = H(s) (6) Among the most widely used filters are linear shift-ivariant (LSI) ones. The z-transform provides a convenient representation for signals and filters in DSP. (In short) An alternative representation for the output signal is given by ˜s = h(C)s (7) where h(c) = N−1 n=0 hnCn (Resultant is a circulant matrix) Big Data Analysis with Signal Processing on Graphs 8-17
  18. 18. Graph Filters and Z-Transform In signal processing, a filter is a system H(.) that takes an input signal s and outputs a signal: ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = H(s) (6) Among the most widely used filters are linear shift-ivariant (LSI) ones. The z-transform provides a convenient representation for signals and filters in DSP. (In short) An alternative representation for the output signal is given by ˜s = h(C)s (7) where h(c) = N−1 n=0 hnCn (Resultant is a circulant matrix) Big Data Analysis with Signal Processing on Graphs 8-18
  19. 19. Graph Filters and Z-Transform (cont’d) DSP on Graphs extends the concept of filters to general graphs. Similarly to the extension of the time shift to the graph shift, filters are generalized to graph filters as polynomials in the graph shift , and all LSI graph filters have the form h(A) = L−1 l=0 hlAl (8) In analogy with signal filters, the graph filter output is given by ˜s = h(A)s (9) Big Data Analysis with Signal Processing on Graphs 8-19
  20. 20. Graph Fourier Transform Mathematically, a Fourier transform with respect to a set of operators is the expansion of a signal into a basis of the operators eigen functions. Since in signal processing the operators of interest are filters, DSPG defines the Fourier transform with respect to the graph filters. ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = GFT{s} (10) Big Data Analysis with Signal Processing on Graphs 8-20
  21. 21. Graph Fourier Transform Mathematically, a Fourier transform with respect to a set of operators is the expansion of a signal into a basis of the operators eigen functions. Since in signal processing the operators of interest are filters, DSPG defines the Fourier transform with respect to the graph filters. ˜s = [ ˜s0, ˜s1, . . . , ˜sN−1] T = GFT{s} (10) Big Data Analysis with Signal Processing on Graphs 8-21
  22. 22. Graph Fourier Transform (cont’d) For simplicity, assume that A is diagonalizable and its decomposition is A = V ΛV −1 (11) where the columns vn of the matrix V = [v0 · · · vN−1] ∈ CN×N are the eigenvectors of A and Λ = diag(λ0, . . . , λN−1) are eigenvalues of A In general A can be diagonalized using Jordan decomposition. Big Data Analysis with Signal Processing on Graphs 8-22
  23. 23. Graph Fourier Transform (cont’d) For simplicity, assume that A is diagonalizable and its decomposition is A = V ΛV −1 (11) where the columns vn of the matrix V = [v0 · · · vN−1] ∈ CN×N are the eigenvectors of A and Λ = diag(λ0, . . . , λN−1) are eigenvalues of A In general A can be diagonalized using Jordan decomposition. Big Data Analysis with Signal Processing on Graphs 8-23
  24. 24. Graph Fourier Transform (cont’d) The eigenfunctions of graph filters h(A) are given by the eigenvectors of the graph shift matrix A Since the expansion into the eigenbasis is given by the multiplication with the inverse eigenvector matrix, which always exists, the graph Fourier transform is well defined and computed as ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1] T = V −1 s (12) = Fs (13) where F = V −1 is the graph Fourier transform matrix. Big Data Analysis with Signal Processing on Graphs 8-24
  25. 25. Graph Fourier Transform (cont’d) The eigenfunctions of graph filters h(A) are given by the eigenvectors of the graph shift matrix A Since the expansion into the eigenbasis is given by the multiplication with the inverse eigenvector matrix, which always exists, the graph Fourier transform is well defined and computed as ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1] T = V −1 s (12) = Fs (13) where F = V −1 is the graph Fourier transform matrix. Big Data Analysis with Signal Processing on Graphs 8-25
  26. 26. Graph Fourier Transform (cont’d) The eigenfunctions of graph filters h(A) are given by the eigenvectors of the graph shift matrix A Since the expansion into the eigenbasis is given by the multiplication with the inverse eigenvector matrix, which always exists, the graph Fourier transform is well defined and computed as ˆs = [ ˆs0, ˆs1, . . . , ˆsN−1] T = V −1 s (12) = Fs (13) where F = V −1 is the graph Fourier transform matrix. Big Data Analysis with Signal Processing on Graphs 8-26
  27. 27. Graph Fourier Transform (cont’d) The inverse graph Fourier transform reconstructs the graph signal from is frequency content by combining graph frequency components weighted by the coefficients of the signal’s graph Fourier transform: s = ˆs0v0 + ˆs1v1 + · · · + ˆsN−1vN−1 (14) = F−1 s = V s (15) Big Data Analysis with Signal Processing on Graphs 8-27
  28. 28. Low and High Frequencies on Graphs The values ˆsn in (12) are the signal’s expansion in the eigenvector basis and represent the frequency content of the signal s. The eigenvalues λn of the shift matrix A represent graph frequency content, and the eigenvectors vn represent the corresponding graph frequency component. To conclude, the higher λn, the higher frequency content and vice versa. Big Data Analysis with Signal Processing on Graphs 8-28
  29. 29. Low and High Frequencies on Graphs The values ˆsn in (12) are the signal’s expansion in the eigenvector basis and represent the frequency content of the signal s. The eigenvalues λn of the shift matrix A represent graph frequency content, and the eigenvectors vn represent the corresponding graph frequency component. To conclude, the higher λn, the higher frequency content and vice versa. Big Data Analysis with Signal Processing on Graphs 8-29
  30. 30. Low and High Frequencies on Graphs The values ˆsn in (12) are the signal’s expansion in the eigenvector basis and represent the frequency content of the signal s. The eigenvalues λn of the shift matrix A represent graph frequency content, and the eigenvectors vn represent the corresponding graph frequency component. To conclude, the higher λn, the higher frequency content and vice versa. Big Data Analysis with Signal Processing on Graphs 8-30
  31. 31. Low and High Frequencies on Graphs The values ˆsn in (12) are the signal’s expansion in the eigenvector basis and represent the frequency content of the signal s. The eigenvalues λn of the shift matrix A represent graph frequency content, and the eigenvectors vn represent the corresponding graph frequency component. To conclude, the higher λn, the higher frequency content and vice versa. Big Data Analysis with Signal Processing on Graphs 8-31
  32. 32. Frequency Response of Graph Filters In addition to expressing the frequency content of graph signals, the graph Fourier transform also characterizes the effect of filters on the frequency content of signals. The filtering operation ˜s = h(A)s can be written using h(A) = L−1 l=0 hlAl and ˆs = V −1 s as follows ˜s = h(A)s = h(F−1 AF)s = F−1 h(Λ)Fs (16) where h(Λ) is a diagonal matrix with values h(λn) = L−1 l=0 hlλl n As a result, ˜s = h(A)s ⇔ Fs = h(Λ)s (17) Big Data Analysis with Signal Processing on Graphs 8-32
  33. 33. Frequency Response of Graph Filters In addition to expressing the frequency content of graph signals, the graph Fourier transform also characterizes the effect of filters on the frequency content of signals. The filtering operation ˜s = h(A)s can be written using h(A) = L−1 l=0 hlAl and ˆs = V −1 s as follows ˜s = h(A)s = h(F−1 AF)s = F−1 h(Λ)Fs (16) where h(Λ) is a diagonal matrix with values h(λn) = L−1 l=0 hlλl n As a result, ˜s = h(A)s ⇔ Fs = h(Λ)s (17) Big Data Analysis with Signal Processing on Graphs 8-33
  34. 34. Frequency Response of Graph Filters In addition to expressing the frequency content of graph signals, the graph Fourier transform also characterizes the effect of filters on the frequency content of signals. The filtering operation ˜s = h(A)s can be written using h(A) = L−1 l=0 hlAl and ˆs = V −1 s as follows ˜s = h(A)s = h(F−1 AF)s = F−1 h(Λ)Fs (16) where h(Λ) is a diagonal matrix with values h(λn) = L−1 l=0 hlλl n As a result, ˜s = h(A)s ⇔ Fs = h(Λ)s (17) Big Data Analysis with Signal Processing on Graphs 8-34
  35. 35. Frequency Response of Graph Filters (cont’d) That is, the frequency content of a filtered signal is modified by multiplying its frequency content element-wise by h(λn) . These values represent the graph frequency response of the graph filter. The relation is a generalization of the classical convolution theorem to graphs: filtering a graph signal in the graph domain is equivalent in the frequency domain to multiplying the signal spectrum by the frequency response of the graph filter. ˜s = h(A)s ⇔ Fs = h(Λ)s Big Data Analysis with Signal Processing on Graphs 8-35
  36. 36. Frequency Response of Graph Filters (cont’d) That is, the frequency content of a filtered signal is modified by multiplying its frequency content element-wise by h(λn) . These values represent the graph frequency response of the graph filter. The relation is a generalization of the classical convolution theorem to graphs: filtering a graph signal in the graph domain is equivalent in the frequency domain to multiplying the signal spectrum by the frequency response of the graph filter. ˜s = h(A)s ⇔ Fs = h(Λ)s Big Data Analysis with Signal Processing on Graphs 8-36
  37. 37. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-37
  38. 38. Product Graphs Consider two graphs G1 = (V1, A1) and G2 = (V2, A2) with |V1| = N1 and |V2| = N2 nodes, respectively. The product graph, denoted by , of G1 and G2 is the graph G = G1 G2 = (V, A ) (18) with |V| = N1N2 and dim(A ) = N1N2 × N1N2. Big Data Analysis with Signal Processing on Graphs 8-38
  39. 39. Common Product Graphs Types 1. Kronecker product A⊗ = A1 ⊗ A2 (19) Example: If we have two matrices B ∈ CM×N and C ∈ CK×L , then the Kronecker product is defined as follows B ⊗ C =     b1,1C ... bM,1C · · · ... · · · b1,M C ... bM,M C     ∈ CMK×NL (20) Big Data Analysis with Signal Processing on Graphs 8-39
  40. 40. Common Product Graphs Types 1. Cartesian product A× = A1 ⊗ IN2 + IN1 ⊗ A2 (21) 2. Strong product A = A1 ⊗ A2 + A1 ⊗ IN2 + IN1 ⊗ A2 (22) Big Data Analysis with Signal Processing on Graphs 8-40
  41. 41. Examples on Product Graphs Big Data Analysis with Signal Processing on Graphs 8-41
  42. 42. Examples on Product Graphs Big Data Analysis with Signal Processing on Graphs 8-42
  43. 43. Examples on Product Graphs Big Data Analysis with Signal Processing on Graphs 8-43
  44. 44. Notes on Product Graphs Big Data Analysis with Signal Processing on Graphs 8-44
  45. 45. Signal Processing on Product Graphs The computation of filtering and Fourier transform on graphs and improve algorithms, data storage, and memory access for large data sets can modularized thanks to graph products. Such as Filtering Computation complexity: O(N2 ) =⇒ O(N(N1 + N2)) Fourier transform Computation complexity: O(N3 ) =⇒ O(N3 1 + N3 2 ) Big Data Analysis with Signal Processing on Graphs 8-45
  46. 46. Signal Processing on Product Graphs The computation of filtering and Fourier transform on graphs and improve algorithms, data storage, and memory access for large data sets can modularized thanks to graph products. Such as Filtering Computation complexity: O(N2 ) =⇒ O(N(N1 + N2)) Fourier transform Computation complexity: O(N3 ) =⇒ O(N3 1 + N3 2 ) Big Data Analysis with Signal Processing on Graphs 8-46
  47. 47. Signal Processing on Product Graphs The computation of filtering and Fourier transform on graphs and improve algorithms, data storage, and memory access for large data sets can modularized thanks to graph products. Such as Filtering Computation complexity: O(N2 ) =⇒ O(N(N1 + N2)) Fourier transform Computation complexity: O(N3 ) =⇒ O(N3 1 + N3 2 ) Big Data Analysis with Signal Processing on Graphs 8-47
  48. 48. Contents Introduction Fundamentals of Graph Theory DSP on Graphs Graph Products Applications Big Data Analysis with Signal Processing on Graphs 8-48
  49. 49. Applications Like-wise traditional DSP problems: Data compression Fourier transform or through wavelet expansions, or adaptive filter design Detection of corrupted data High pass filter Big Data Analysis with Signal Processing on Graphs 8-49
  50. 50. Applications Like-wise traditional DSP problems: Data compression Fourier transform or through wavelet expansions, or adaptive filter design Detection of corrupted data High pass filter Big Data Analysis with Signal Processing on Graphs 8-50
  51. 51. Applications Like-wise traditional DSP problems: Data compression Fourier transform or through wavelet expansions, or adaptive filter design Detection of corrupted data High pass filter Big Data Analysis with Signal Processing on Graphs 8-51
  52. 52. Challenges of Big Data While there is no single, universally agreed upon set of properties that define big data. Some of the commonly mentioned ones are volume, velocity, and variety of data. Big Data Analysis with Signal Processing on Graphs 8-52
  53. 53. Challenges of Big Data While there is no single, universally agreed upon set of properties that define big data. Some of the commonly mentioned ones are volume, velocity, and variety of data. Big Data Analysis with Signal Processing on Graphs 8-53
  54. 54. Challenges of Big Data Some of the commonly mentioned ones are volume, velocity, and variety of data. First of all, the sheer volume of data to be processed requires efficient distributed and scalable storage, access, and processing. High velocity of new data arrival demands fast algorithms to prevent bottlenecks and explosion of the data volume and to extract valuable information from the data and incorporate it into the decision-making process in real time. (FFT in DSP) Finally, collected data sets contain information in all varieties and forms, including numerical, textual, and visual data. To generalize data analysis techniques to diverse data sets, we need a common representation framework for data sets and their structure. Big Data Analysis with Signal Processing on Graphs 8-54
  55. 55. Challenges of Big Data Some of the commonly mentioned ones are volume, velocity, and variety of data. First of all, the sheer volume of data to be processed requires efficient distributed and scalable storage, access, and processing. High velocity of new data arrival demands fast algorithms to prevent bottlenecks and explosion of the data volume and to extract valuable information from the data and incorporate it into the decision-making process in real time. (FFT in DSP) Finally, collected data sets contain information in all varieties and forms, including numerical, textual, and visual data. To generalize data analysis techniques to diverse data sets, we need a common representation framework for data sets and their structure. Big Data Analysis with Signal Processing on Graphs 8-55
  56. 56. Challenges of Big Data Some of the commonly mentioned ones are volume, velocity, and variety of data. First of all, the sheer volume of data to be processed requires efficient distributed and scalable storage, access, and processing. High velocity of new data arrival demands fast algorithms to prevent bottlenecks and explosion of the data volume and to extract valuable information from the data and incorporate it into the decision-making process in real time. (FFT in DSP) Finally, collected data sets contain information in all varieties and forms, including numerical, textual, and visual data. To generalize data analysis techniques to diverse data sets, we need a common representation framework for data sets and their structure. Big Data Analysis with Signal Processing on Graphs 8-56
  57. 57. Challenges of Big Data Some of the commonly mentioned ones are volume, velocity, and variety of data. First of all, the sheer volume of data to be processed requires efficient distributed and scalable storage, access, and processing. High velocity of new data arrival demands fast algorithms to prevent bottlenecks and explosion of the data volume and to extract valuable information from the data and incorporate it into the decision-making process in real time. (FFT in DSP) Finally, collected data sets contain information in all varieties and forms, including numerical, textual, and visual data. To generalize data analysis techniques to diverse data sets, we need a common representation framework for data sets and their structure. Big Data Analysis with Signal Processing on Graphs 8-57
  58. 58. Thank You! Big Data Analysis with Signal Processing on Graphs 8-58

×