Successfully reported this slideshow.
Upcoming SlideShare
×

# tensor-decomposition

47,674 views

Published on

• Full Name
Comment goes here.

Are you sure you want to Yes No
• You might get some help from ⇒ www.HelpWriting.net ⇐ Success and best regards!

Are you sure you want to  Yes  No
• There is a useful site for you that will help you to write a perfect and valuable essay and so on. Check out, please ⇒ www.WritePaper.info ⇐

Are you sure you want to  Yes  No
• Get Result In Week One! Thank you, thank you, thank you! I've been plagued with small breasts for 30 years and even when I thought I couldn't do anything, your guide helped me to grow 1 cup size! I'm so pleased and I started seeing results in the first week. Thanks again Jenny, your work is amazing!! ★★★ https://dwz1.cc/YYZPZbuh

Are you sure you want to  Yes  No
• The #1 Woodworking Resource With Over 16,000 Plans, Download 50 FREE Plans... ➜➜➜ https://url.cn/xFeBN0O4

Are you sure you want to  Yes  No
• NOTE TO READERS: Do NOT Buy Any Shed Plans Until You've Seen This... ✤✤✤ https://url.cn/sIEM11KJ

Are you sure you want to  Yes  No

### tensor-decomposition

1. 1. . Introduction to Tensors PFI Seminar Kenta OONO 2012/07/11Kenta OONOIntroduction to Tensors
2. 2. Self Introduction • Kenta OONO (Twitter:@delta2323 ) • PSS(Professional and Support Service) Division • Jubatus Project • msgpack-idl, generation of clients • Survey and Prototyping Team • 2010 PFI Summer Intern → 2011 EngineerKenta OONOIntroduction to Tensors
3. 3. Agenda • What is Tensor? • Decomposition of Matrices and Tensors • Symmetry Parametrized by Young DiagramKenta OONOIntroduction to Tensors
4. 4. Agenda • What is Tensor? • Decomposition of Matrices and Tensors • Symmetry Parametrized by Young DiagramKenta OONOIntroduction to Tensors
5. 5. Vector as a Special Case of Tensor Consider a n dimensional vector (Below is the case n = 3)   7   v = 4 (1) 9 When we write this vector as v = (vi ), we have v1 = 7, v2 = 4, and v3 = 9. As v has One index, this vector is regarded as Rank 1 Tensor. Note • We use 1-index notation, not 0-index. • We somtimes use rank in a diﬀerent meaning, (also in the context of tensors).Kenta OONOIntroduction to Tensors
6. 6. Matrix as a Special Case of Tensor Consider a n × n Matrix (Below is the case n = 3).   7 10 3   A = 4 −1 −2 (2) 9 4 5 We write this matrix as A = (aij )1≤i,j≤3 . • e.g. a11 = 7, a23 = −2, a32 = 4. A has Two indices (namely i and j) and these indeices run through 1 to 3. A can be regarded as Rank 2 Tensor. → What happens if the number of indices are not just 1 or 2 ?Kenta OONOIntroduction to Tensors
7. 7. An Example of Rank 3 Tensor Consider the set of real numbers X = (xijk )1≤i,j,k≤3 . As i, j, k run throuth 1 to 3 independently, this set contains 3 × 3 × 3 = 27 elements. If we ﬁx the value of i, we can write this set in a matrix form.       7 10 3 10 2 1 5 −10 18       x1∗∗ = 4 −1 −2 , x2∗∗ =  3 −1 0 , x3∗∗ = −4 1 −2  9 4 5 8 1 4 9 −4 −10 (3) Note • In some ﬁeld, we concatenate these matrix and name it X(1) • This is called Flattening or Matricing of X .Kenta OONOIntroduction to Tensors
8. 8. Is This a Multidimensional Array? We can realize a tensor as a multiple dimensional array in most programming languages. PseudoCode(C like): int[3][3][3] x; x[1][1][2] = -1;Kenta OONOIntroduction to Tensors
9. 9. Inner Product of Tensors Let X = (xijk ), Y = (yijk ) be two rank 3 tensors and G = (g ij ) be a symmetric (i.e. g ij = g ji ) and positive deﬁnite matrix. We can introduce an inner product of X and Y by: ∑ n < X , Y >= g ai g bj g ck xabc yijk (4) a,b,c,i,j,k=1 Note: • We can similarly deﬁne an inner product of two arbitrary rank tensor • X and Y must have same rank.Kenta OONOIntroduction to Tensors
10. 10. An Example of Inner Product (Vector) Let’s consider the case of rank 1 tensor ( = vector). We can write X = (xi ), Y = (yi ). And we set G = (g ij ) an identity matrix:   1 ··· 0 . . G = . 1 . . . (5) 0 ··· 1 In this case, this deﬁnition of inner product reduce to ordinal inner product of two vectors, namely, ∑ ∑ < X , Y >= g ai xa yi = xi yi (6) a,i iKenta OONOIntroduction to Tensors
11. 11. An Example of Tensor Suppose we have a (smooth) function f : R3 → R. We can derive tensors of arbitrary rank from this function. Taking Gradient, we obtain rank 1 tensor. (Please replace (1, 2, 3) with (x , y , z) and vice versa.  ∂f   ∂x  ∇f = (∇i f ) =  ∂y  ∂f (7) ∂f ∂z Similarly, Hessian is an example of rank 2 tensor.   ∂2f ∂2f ∂2f  ∂x2∂x ∂x ∂y ∂x ∂z  H(f ) = (Hij (f )) =  ∂ f ∂2f ∂2f  (8)  ∂y ∂x ∂y ∂y ∂y ∂z  ∂2f ∂2f ∂2f ∂z∂x ∂z∂y ∂z∂zKenta OONOIntroduction to Tensors
12. 12. Supersymmetry Property By taking derivative, we can create an arbitrary rank of tensors. These tensors are (Super)Symmetric in the sense that interchange of any two indices remains the tensor identical. For example, fx y z = fx zy = fy x z = fy zx = fzx y = fzy x . (9) Note: • Afterward, we write this symmetry as .Kenta OONOIntroduction to Tensors
13. 13. Agenda • What is Tensor? • Decomposition of Matrices and Tensors • Symmetry Parametrized by Young DiagramKenta OONOIntroduction to Tensors
14. 14. Decompose! It is fundamental in mathematics to decompose some object into simpler, easier-to-handle objects. ∑ √ • Fourier Expansion : f = an exp( −1nx ) n∈Z ∑ • Legendre Polynomial : P = ak Pk k 1 dk 2 • where Pk (x ) = (x − 1)k k!2k dx k • Jordan-H¨lder : e = G0 ◁ G1 ◁ G2 ◁ · · · ◁ Gn−1 ◁ Gn = G oKenta OONOIntroduction to Tensors
15. 15. Decomposition of Matrix There are many theories regarding the decomposition (or, factorization) of matrices. • SVD • LU • QR • Cholesky • Jordan • Diagonalization ... Although we have theories of factorization of Tensors (e.g. Tucker Decomposition, higher-order SVD etc.) We do NOT go this direction. Instead, we consider decomposition of matrix into Summation of matrix.Kenta OONOIntroduction to Tensors
16. 16. Example: Projection to AxisKenta OONOIntroduction to Tensors
17. 17. Example of Decomposition of Matrix We can decompose matrix into Symmetric part and Antisymmetric part. Example:       7 10 3 7 7 6 0 3 −3       4 −1 −2 = 7 −1 1 + −3 0 −3 9 4 5 6 1 5 3 3 0 Symmetric Part Antisymmetric Part A Asym Aanti (10)Kenta OONOIntroduction to Tensors
18. 18. How to Calculate Sym. and Antisym. Part We can calculate the symmetic and antisymmetric part by simple calculation (Exercise!).     7 10 3 7 4 9 1     Asym = 4 −1 −2 + 10 −1 4 2 9 4 5 3 −2 5 (11) sym 1 aij = (aij + aji ) 2     7 10 3 7 4 9 1     Aanti = 4 −1 −2 − 10 −1 4 2 9 4 5 3 −2 5 (12) 1 anti aij = (aij − aji ) 2Kenta OONOIntroduction to Tensors
19. 19. Picture of Projection to Sym. and Antisym. PartKenta OONOIntroduction to Tensors
20. 20. Notation Using Young Diagram We can symbolize symmetry and antisymmetry with Young Diagram       7 10 3 7 7 6 0 3 −3       4 −1 −2 = 7 −1 1 + −3 0 −3 9 4 5 6 1 5 3 3 0 (13) A Asym Aanti The number of boxes indicates rank of the tensor (one box for vector and two boxes for matrix).Kenta OONOIntroduction to Tensors
21. 21. Agenda • What is Tensor? • Decomposition of Matrices and Tensors • Symmetry Parametrized by Young Diagram Note: • From now on, we concentrate on Rank 3 Tensors (i.e. k = 3). • And we assume that n = 3, that is, indices run from 1 to 3.Kenta OONOIntroduction to Tensors
22. 22. Symmetrizing Operator We consider the transformation of tensor: T : X = (xijk ) → X ′ = (xijk ), ′ (14) ′ where xijk = 1 (xijk + xikj + xjik + xjki + xkij + xkji ) 6 (15) = x(ijk) .Kenta OONOIntroduction to Tensors
23. 23. Young Diagram and Symmetry of Tensor (Sym. Case) Let X = (x123 ) be a tensor of rank 3, we call X Has a Symmetry of , if interchange of any of two indices doesn’t change each entry of X . Example: • If X has a symmetry of , then x112 = x121 = x211 (16) x123 = x132 = x213 = x231 = x312 = x321 etc... • Symmetric matrices have a symmetry ofKenta OONOIntroduction to Tensors
24. 24. Property of T • For all X , T (X ) has symmetry of . • If X has symmetry of , then T (X ) = X . • For all X , T (T (X )) = T (X ).Kenta OONOIntroduction to Tensors
25. 25. Antisymmetrizing Operator Next, we consider Antisymmetrization of tensors: T : X = (xijk ) → X ′ = (xijk ), ′ (17) ′ where xijk = 1 (xijk − xikj − xjik + xjki + xkij − xkji ) 6 (18) = x[ijk] .Kenta OONOIntroduction to Tensors
26. 26. Young Diagram and Symmetry of Tensor (Antisym. Case) Let X = (x123 ) be a tensor of rank k, we call X has a symmetry of if interchange of any of two indices change only the sign of each entry of X . Example. • If X has a symmetry of , then • x123 = −x132 = −x213 = x231 = x312 = −x321 • x112 = 0, because x112 must be equal to −x112 . • Antisymmetric matrix has a symmetry ofKenta OONOIntroduction to Tensors
27. 27. Orthogonality of Projection T and T are orthogonal in the sense T ◦ T (X ) = T ◦ T (X ) = 0. (19)Kenta OONOIntroduction to Tensors
28. 28. Counting Dimension How many dimension do the vector space consisting of tensors ... • with symmetry have ? • with symmetry have ?Kenta OONOIntroduction to Tensors
29. 29. Dimensions of each subvector spaces. If n = 3 (that is, i, j and k runs through 1 to 3), then each dimension is as follows. But dimension of total space is 3 × 3 × 3 = 27. This is larger than 1 + 10 = 11.Kenta OONOIntroduction to Tensors
30. 30. Young Diagram Young Diagram is an arrangent of boxes of the same size so that: • two adjacent boxes share one of their side. • the number of boxes in each row is non-increasing. • the number of boxes in each column is non-increasing. Here is an example of Young Diagram with 10 boxes. (20) English Notation French NotationKenta OONOIntroduction to Tensors
31. 31. Decomposition of Tensor (of Rank 3) We have three types of Young Diagram which have three boxes, namely, (21) , , and Symmetric Antisymmetric ??? → What symmetry does represent?Kenta OONOIntroduction to Tensors
32. 32. Coeﬃcients of Symmetrizers We list coneﬃents of each operator in a tabular form. Coeﬃcient of xijk ijk ikj jik jki kij kji 1/6 1/6 1/6 1/6 1/6 1/6 1/6 -1/6 -1/6 1/6 1/6 -1/6 Sum 1Kenta OONOIntroduction to Tensors
33. 33. Coeﬃcients of Symmetrizers Coeﬃcient of xijk ijk ikj jik jki kij kji 1/6 1/6 1/6 1/6 1/6 1/6 1/6 -1/6 -1/6 1/6 1/6 -1/6 4/6 -2/6 -2/6 Sum 1Kenta OONOIntroduction to Tensors
34. 34. Symmetrizer T We consider the transformation of tensor: T : X = (xijk ) → X ′ = (xijk ), ′ (22) ′ where xijk = 1 (2xijk − xjki − xkij ). 3Kenta OONOIntroduction to Tensors
35. 35. Property of T • For all X , T (X ) has symmetry of . • If X has symmetry of , then T (X ) = X . ( ) • For all X , T T (X ) = T (X ).Kenta OONOIntroduction to Tensors
36. 36. Tensors with Symmetry of Let X = (xijk ) be a tensor of rank 3. We say X Has a Symmetry of , if 1 xijk = (2xijk − xjki − xkij ) (23) 3 for all 1 ≤ i, j, k ≤ n.Kenta OONOIntroduction to Tensors
37. 37. Where This Equation ”Symmetric” ? Remember that X has a symmetry of , if 1 xijk = (2xijk − xjki − xkij ) (24) 3 for all 1 ≤ i, j, k ≤ n. This equation can be transormed as follows : 1 (2xijk − xjki − xkij ) xijk = 3 (25) ⇔ 3xijk = 2xijk − xjki − xkij ⇔ xijk + xjki + xkij = 0Kenta OONOIntroduction to Tensors
38. 38. Projection to Each Symmetrized TensorsKenta OONOIntroduction to Tensors
39. 39. Decomposition of Higher Rank Tensors Higher rank tensors are also decompose into symmetric tensors parametrized by Young Diagram. ⊕ ⊕ Rn×n×n×n×n = ⊕ ⊕ ⊕ ⊕ (26)Kenta OONOIntroduction to Tensors
40. 40. Summary • Tensors as a Generalization of Vectors and Matrices. • Decomposition of Matrices and Tensors • Symmetry of Tensors Parametrized by Young Diagrams.Kenta OONOIntroduction to Tensors
41. 41. References • T G. Kolda and B W. Bader, Tensor Decompositions and Applications • 2009, SIAM Review Vol. 51, No 3, pp 455-500 • D S. Watkins, Fundamentals of Matrix Computations 3rd. ed. • 2010, A Wiley Series of Texts, Monographs and Tracts • W. Fulton and J Harris, Representation Theory • 1991, Graduate Texts in Mathematics Readins in Mathematics. • And Google ”Representation Theory”, ”Symmetric Group”, ”Young Diagram” and so on.Kenta OONOIntroduction to Tensors