Successfully reported this slideshow.
Your SlideShare is downloading. ×

Tensor Decomposition with Missing Indices

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 18 Ad
Advertisement

More Related Content

Similar to Tensor Decomposition with Missing Indices (20)

More from Yuto Yamaguchi (13)

Advertisement

Recently uploaded (20)

Tensor Decomposition with Missing Indices

  1. 1. Tensor  Decomposi-on  with   Missing  Indices Yuto  Yamaguchi  and  Kohei  Hayashi 17/08/22 IJCAI2017@Melbourne 1
  2. 2. Tensor  data 17/08/22 IJCAI2017@Melbourne 2 #     # (userA,    #movie,    Melbourne):  1   (userB,    #tennis,    Sydney):    2   (userC,    #dinner,  Canberra):    1   (userB,    #beer,    Brisbane):    1   (userA,    #dinner,  Melbourne):  2   e.g.,  TwiNer  data  (user,  hashtag,  loca-on) Tensor  data  =  mul--­‐dimensional  data value
  3. 3. Tensor  decomposi-on 17/08/22 IJCAI2017@Melbourne 3 e.g.,  CP  decomposi-on  [Carroll  and  Chang,  1970] + +        … = Applica-ons   •  Recommenda-ons,  noise  reduc-on,  data  compression,  …   ˆXijk = UirVjrWkr r ∑ X V:,  1 U:,  1 W:,  1 V:,  2 U:,  2 W:,  2
  4. 4. [Our  problem]   what  if  indices  are  missing? 17/08/22 IJCAI2017@Melbourne 4 #     # (userA,    #movie,    Melbourne):  1   (userB,    #tennis,    Sydney):    2   (userC,    #dinner,  Canberra):    1   (userB,    #beer,    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐):    1   (userA,    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐,  Melbourne):  2   Conven5onal  tensor  decomposi5on  algorithms   do  not  apply  to  these  “incomplete  samples”  L value
  5. 5. [Our  problem]   what  if  indices  are  missing? 17/08/22 IJCAI2017@Melbourne 5 #     # (userA,    #movie,    Melbourne):  1   (userB,    #tennis,    Sydney):    2   (userC,    #dinner,  Canberra):    1   (userB,    #beer,    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐):    1   (userA,    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐,  Melbourne):  2   Conven5onal  tensor  decomposi5on  algorithms   do  not  apply  to  these  “incomplete  samples”  L value Values  are  not  missing
  6. 6. PROPOSED  MODEL 17/08/22 IJCAI2017@Melbourne 6
  7. 7. Basic  idea 17/08/22 IJCAI2017@Melbourne 7 (userA,  #movie,    Melbourne)   (userB,  #tennis,    Sydney)   (userC,  #dinner,  Canberra)   (userB,  #beer,  Brisbane)   (userA,  #dinner,  Melbourne)   + +        … e.g.,  CPD infer construct decompose Solve  tensor  decomposi5on  and  missing  indices  inference   repeatedly
  8. 8. Proposed  model  (1/2) 17/08/22 IJCAI2017@Melbourne 8 Handle  indices  as  unobserved  variables ˆin ∈ 1,2,…I,φ{ } Observed  (can  be  missing)  indices True  (unobserved)  indices missing Tensor  elements Decomposi5on   parameters [3rd-­‐order  case]
  9. 9. Proposed  model  (2/2) 17/08/22 IJCAI2017@Melbourne 9 1.  Generate  decomposi-on  parameters  depending  on  the              decomposi-on  model   Θ = U,V,W{ } Uir = N ⋅ 0, 1 λ " # $ % & ' for  all  i  and  r e.g.,  CPD
  10. 10. Proposed  model  (2/2) 17/08/22 IJCAI2017@Melbourne 10 2.  Generate  N  indices  (in,  jn,  kn)   Delta  if  not  missing Uniform  if  missing in ~
  11. 11. Proposed  model  (2/2) 17/08/22 IJCAI2017@Melbourne 11 3.  Generate  N  tensor  elements  depending  on  decomposi-on  model   e.g.,  CPD ˆXin jnkn = UinrVjnrWknr r ∑
  12. 12. Proposed  model  is  a  natural  extension  of   the  conven-onal  tensor  decomposi-on 17/08/22 IJCAI2017@Melbourne 12 where MLE  Θ  of  the  proposed  model
  13. 13. Parameter  inference Varia-onal  MAP-­‐EM  algorithm       •  E-­‐step   – Missing  indices  are  inferred  using  learnt  tensor   decomposi-on   •  M-­‐step   – Tensor  decomposi-on  is  learnt  using  inferred   indices 17/08/22 IJCAI2017@Melbourne 13 See  the  paper  for  details  if  interested  J
  14. 14. Time  Complexity  (Mth-­‐order  tensor) 17/08/22 IJCAI2017@Melbourne 14 Proposed  algorithm  for  CPD Conven-onal  CPD N Nm - R Im :  #  of  samples :  #  of  missing  indices  for  mth  mode :  #  of  latent  dimensions :  #  of  dimensions  for  mth  mode Only  addi5onal  term
  15. 15. EXPERIMENTS 17/08/22 IJCAI2017@Melbourne 15
  16. 16. Compared  algorithms 17/08/22 IJCAI2017@Melbourne 16 [MAP-­‐EM]:    Proposed  algo.  with  q  inferred     [Uniform]:    Proposed  algo.  with  q  fixed  as  uniform     [Prior]:      Proposed  algo.  with  q  fixed  as  data  histogram     [Minimal]:    CPD  with  only  complete  samples     [Complete]:  CPD  with  only  complete  modes     [CMTF]:      Coupled  matrix  tensor  factoriza-on  [Acar+,  2011] Approx.  distribu5on  on  varia5onal  inference Proposed Baselines
  17. 17. Results 17/08/22 IJCAI2017@Melbourne 17 Lower  beNer Lower  beNer Upper  beNer Proposed  model  (red)  works  well  if   •  the  number  of  samples  is  large,  or   •  missing  ra-o  is  not  very  large Synthe5c  data  generated  by  our  model TwiZer  data   (user,  hashtag,  loca5on) sample  size  large  (n=10) sample  size  small  (n=1)
  18. 18. Summary •  [New problem] –  Defined a new tensor decomposition problem where the indices are partially missing •  [Model] –  Proposed a probabilistic generative model to handle missing indices •  [Algorithm] –  Developed a parameter inference algorithm 17/08/22 IJCAI2017@Melbourne 18 Github: yamaguchiyuto/missing_tensor_decomposition

×