Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tensor Decomposition with Missing Indices

1,505 views

Published on

IJCAI 2017 paper presentation

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Tensor Decomposition with Missing Indices

  1. 1. Tensor  Decomposi-on  with   Missing  Indices Yuto  Yamaguchi  and  Kohei  Hayashi 17/08/22 IJCAI2017@Melbourne 1
  2. 2. Tensor  data 17/08/22 IJCAI2017@Melbourne 2 #     # (userA,    #movie,    Melbourne):  1   (userB,    #tennis,    Sydney):    2   (userC,    #dinner,  Canberra):    1   (userB,    #beer,    Brisbane):    1   (userA,    #dinner,  Melbourne):  2   e.g.,  TwiNer  data  (user,  hashtag,  loca-on) Tensor  data  =  mul--­‐dimensional  data value
  3. 3. Tensor  decomposi-on 17/08/22 IJCAI2017@Melbourne 3 e.g.,  CP  decomposi-on  [Carroll  and  Chang,  1970] + +        … = Applica-ons   •  Recommenda-ons,  noise  reduc-on,  data  compression,  …   ˆXijk = UirVjrWkr r ∑ X V:,  1 U:,  1 W:,  1 V:,  2 U:,  2 W:,  2
  4. 4. [Our  problem]   what  if  indices  are  missing? 17/08/22 IJCAI2017@Melbourne 4 #     # (userA,    #movie,    Melbourne):  1   (userB,    #tennis,    Sydney):    2   (userC,    #dinner,  Canberra):    1   (userB,    #beer,    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐):    1   (userA,    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐,  Melbourne):  2   Conven5onal  tensor  decomposi5on  algorithms   do  not  apply  to  these  “incomplete  samples”  L value
  5. 5. [Our  problem]   what  if  indices  are  missing? 17/08/22 IJCAI2017@Melbourne 5 #     # (userA,    #movie,    Melbourne):  1   (userB,    #tennis,    Sydney):    2   (userC,    #dinner,  Canberra):    1   (userB,    #beer,    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐):    1   (userA,    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐,  Melbourne):  2   Conven5onal  tensor  decomposi5on  algorithms   do  not  apply  to  these  “incomplete  samples”  L value Values  are  not  missing
  6. 6. PROPOSED  MODEL 17/08/22 IJCAI2017@Melbourne 6
  7. 7. Basic  idea 17/08/22 IJCAI2017@Melbourne 7 (userA,  #movie,    Melbourne)   (userB,  #tennis,    Sydney)   (userC,  #dinner,  Canberra)   (userB,  #beer,  Brisbane)   (userA,  #dinner,  Melbourne)   + +        … e.g.,  CPD infer construct decompose Solve  tensor  decomposi5on  and  missing  indices  inference   repeatedly
  8. 8. Proposed  model  (1/2) 17/08/22 IJCAI2017@Melbourne 8 Handle  indices  as  unobserved  variables ˆin ∈ 1,2,…I,φ{ } Observed  (can  be  missing)  indices True  (unobserved)  indices missing Tensor  elements Decomposi5on   parameters [3rd-­‐order  case]
  9. 9. Proposed  model  (2/2) 17/08/22 IJCAI2017@Melbourne 9 1.  Generate  decomposi-on  parameters  depending  on  the              decomposi-on  model   Θ = U,V,W{ } Uir = N ⋅ 0, 1 λ " # $ % & ' for  all  i  and  r e.g.,  CPD
  10. 10. Proposed  model  (2/2) 17/08/22 IJCAI2017@Melbourne 10 2.  Generate  N  indices  (in,  jn,  kn)   Delta  if  not  missing Uniform  if  missing in ~
  11. 11. Proposed  model  (2/2) 17/08/22 IJCAI2017@Melbourne 11 3.  Generate  N  tensor  elements  depending  on  decomposi-on  model   e.g.,  CPD ˆXin jnkn = UinrVjnrWknr r ∑
  12. 12. Proposed  model  is  a  natural  extension  of   the  conven-onal  tensor  decomposi-on 17/08/22 IJCAI2017@Melbourne 12 where MLE  Θ  of  the  proposed  model
  13. 13. Parameter  inference Varia-onal  MAP-­‐EM  algorithm       •  E-­‐step   – Missing  indices  are  inferred  using  learnt  tensor   decomposi-on   •  M-­‐step   – Tensor  decomposi-on  is  learnt  using  inferred   indices 17/08/22 IJCAI2017@Melbourne 13 See  the  paper  for  details  if  interested  J
  14. 14. Time  Complexity  (Mth-­‐order  tensor) 17/08/22 IJCAI2017@Melbourne 14 Proposed  algorithm  for  CPD Conven-onal  CPD N Nm - R Im :  #  of  samples :  #  of  missing  indices  for  mth  mode :  #  of  latent  dimensions :  #  of  dimensions  for  mth  mode Only  addi5onal  term
  15. 15. EXPERIMENTS 17/08/22 IJCAI2017@Melbourne 15
  16. 16. Compared  algorithms 17/08/22 IJCAI2017@Melbourne 16 [MAP-­‐EM]:    Proposed  algo.  with  q  inferred     [Uniform]:    Proposed  algo.  with  q  fixed  as  uniform     [Prior]:      Proposed  algo.  with  q  fixed  as  data  histogram     [Minimal]:    CPD  with  only  complete  samples     [Complete]:  CPD  with  only  complete  modes     [CMTF]:      Coupled  matrix  tensor  factoriza-on  [Acar+,  2011] Approx.  distribu5on  on  varia5onal  inference Proposed Baselines
  17. 17. Results 17/08/22 IJCAI2017@Melbourne 17 Lower  beNer Lower  beNer Upper  beNer Proposed  model  (red)  works  well  if   •  the  number  of  samples  is  large,  or   •  missing  ra-o  is  not  very  large Synthe5c  data  generated  by  our  model TwiZer  data   (user,  hashtag,  loca5on) sample  size  large  (n=10) sample  size  small  (n=1)
  18. 18. Summary •  [New problem] –  Defined a new tensor decomposition problem where the indices are partially missing •  [Model] –  Proposed a probabilistic generative model to handle missing indices •  [Algorithm] –  Developed a parameter inference algorithm 17/08/22 IJCAI2017@Melbourne 18 Github: yamaguchiyuto/missing_tensor_decomposition

×