When Does Label Propagation Fail? A View from a Network Generative Model

Yuto Yamaguchi
Yuto YamaguchiIndeed.com - Software Engineer at Indeed.com
When	
  Does	
  Label	
  Propaga1on	
  Fail?	
  
A	
  View	
  from	
  a	
  Network	
  Genera1ve	
  Model	
Yuto	
  Yamaguchi	
  and	
  Kohei	
  Hayashi	
17/08/22	
 IJCAI@Melbourne	
 1
Node Classification	
Given	
 Find	
Partially labeled
undirected graph	
Labels of all nodes	
17/08/22	
 IJCAI@Melbourne	
 2
Example:
User profile inference	
Friends	
Soccer	
 Soccer	
Soccer	
Tennis	
Baseball	
???	
What’s his hobby?
Node Classification	
17/08/22	
 IJCAI@Melbourne	
 3
Label Propagation (1/2)	
Propagate neighbors’ labels
Friends	
Soccer	
 Soccer	
Soccer	
Tennis	
Baseline	
???	
Soccer	
 Soccer	
Soccer	
Tennis	
Baseline	
Soccer	
[Zhu+, 03], [Zhou+, 03], etc.	
17/08/22	
 IJCAI@Melbourne	
 4
Label Propagation (2/2)	
Q F;X,Y,λ( )=
1
2
fi − yi 2
2
i=1
N
∑ +
λ
2
xij fi − fj 2
2
j=1
N
∑
i=1
N
∑
Given: adjacency matrix X and labels Y
Find: F = { fi } that minimizes Q
17/08/22	
 IJCAI@Melbourne	
 5	
F ∈ RN x K
Y ∈ {0, 1}N x K	
X ∈ {0, 1}N x N
N: # of nodes
K: # of labels
λ ∈ R+ : user parameter
[Zhu+, 03], [Zhou+, 03], etc.
Cases	
  when	
  LP	
  fails	
  (prac1cally	
  known)	
Different labels
are connected	
 Label ratio is not uniform	
Q. So, do we know why LP fails in these cases?

A. No. Since it’s not a probabilistic model, we
don’t know the assumptions behind the model.
17/08/22	
 IJCAI@Melbourne	
 6	
Edge probability is not uniform
What	
  we	
  do	
  in	
  this	
  work	
1.  Prove	
  a	
  theore1cal	
  rela1onship	
  between	
  LP	
  
and	
  Stochas(c	
  Block	
  Model,	
  which	
  is	
  a	
  well-­‐
studied	
  probabilis1c	
  genera1ve	
  model	
  
2.  Find	
  the	
  assump(ons	
  behind	
  LP	
  through	
  the	
  
assump1ons	
  behind	
  SBM	
  
3.  Show	
  when	
  and	
  why	
  LP	
  fails	
17/08/22	
 IJCAI@Melbourne	
 7
NETWORK	
  GENERATIVE	
  MODELS	
17/08/22	
 IJCAI@Melbourne	
 8
Stochastic Block Model	
Generative process	
 Multinomial	
Bernoulli	
①	
②	
①: Generate cluster assignment for each node

 
(which can be thought of labels)
②: Generate adjacency matrix	
17/08/22	
 IJCAI@Melbourne	
 9	
γ ∈ RK
Π ∈ RKxK
Parameters:
Proposed:
Partially Labeled SBM (PLSBM) 	
Generative process	
①	
②	
③	
②:Generate labels for “labeled nodes” 

 
(α large à yi is more likely to be the same as zi)
Depends on
parameter α	
17/08/22	
 IJCAI@Melbourne	
 10	
γ ∈ RK
Π ∈ RKxK
α ∈ 0,1[ ]
Parameters:
Rela1onships	
  between	
  models	
17/08/22	
 IJCAI@Melbourne	
 11	
SBM	
 PLSBM	
LP	
 Discre1zed	
  LP	
Main	
  result	
  
(next	
  slide)	
No	
  labels	
Con1nuous	
  
relaxa1on
Main Result	
Map estimator Z of PLSBM is identical to the solution of
(discretized) LP when the following conditions hold 
Condition 1: 
Condition 2:
Condition 3: 
Condition 4: (omitted)
17/08/22	
 IJCAI@Melbourne	
 12
Condition 1	
Implication (implicit assumption of LP)

•  Label ratio is uniform	
17/08/22	
 IJCAI@Melbourne	
 13	
Violates this assumption L
Condition 2	
Implication (Implicit assumptions of LP)	

•  Edge probs between the same labels are all the same (μ)
•  Edge probs between different labels are all the same (ν)	
17/08/22	
 IJCAI@Melbourne	
 14	
Violates this assumption L
Condition 3	
Implication (Implicit assumption of LP)

•  Assortative (same labels tend to be connected)
17/08/22	
 IJCAI@Melbourne	
 15	
Violates this assumption L
Experimental results	
17/08/22	
 IJCAI@Melbourne	
 16	
… Come see full results at the poster session J	
Better	
Setups:
1.  Generate datasets by PLSBM
2.  infer labels (Z) by PLSBM, SBM, and LP
3.  Report mean accuracy of 20 trials	
Assortative	
 Disassortative	
Agree with
theoretical results
Summary	
•  Proposed	
  Par1ally-­‐Labeled	
  SBM	
  (PLSBM)	
  
•  Proved	
  the	
  rela1onship	
  between	
  LP	
  and	
  SBM	
  via	
  
PLSBM	
  
•  Showed	
  cases	
  when	
  LP	
  fails	
  
•  Experimental	
  and	
  Theore1cal	
  results	
  agree	
17/08/22	
 IJCAI@Melbourne	
 17	
Github: yamaguchiyuto/plsbm
1 of 17

Recommended

Alexander Sirenko - Query expansion for Question Answering by
Alexander Sirenko - Query expansion for Question AnsweringAlexander Sirenko - Query expansion for Question Answering
Alexander Sirenko - Query expansion for Question AnsweringAlexander Sirenko
461 views12 slides
ESWC SS 2012 - Monday Tutorial 2 Barry Norton: SPARQL 1.1 Query Language by
ESWC SS 2012 - Monday Tutorial 2 Barry Norton: SPARQL 1.1 Query LanguageESWC SS 2012 - Monday Tutorial 2 Barry Norton: SPARQL 1.1 Query Language
ESWC SS 2012 - Monday Tutorial 2 Barry Norton: SPARQL 1.1 Query Languageeswcsummerschool
415 views13 slides
When Does Label Propagation Fail? A View from a Network Generative Model@ERAT... by
When Does Label Propagation Fail? A View from a Network Generative Model@ERAT...When Does Label Propagation Fail? A View from a Network Generative Model@ERAT...
When Does Label Propagation Fail? A View from a Network Generative Model@ERAT...Yuto Yamaguchi
4.5K views23 slides
Bridging Relational Learning Algorithms@ビッグデータ基盤勉強会 by
Bridging Relational Learning Algorithms@ビッグデータ基盤勉強会Bridging Relational Learning Algorithms@ビッグデータ基盤勉強会
Bridging Relational Learning Algorithms@ビッグデータ基盤勉強会Yuto Yamaguchi
4.5K views41 slides
Tensor Decomposition with Missing Indices by
Tensor Decomposition with Missing IndicesTensor Decomposition with Missing Indices
Tensor Decomposition with Missing IndicesYuto Yamaguchi
4.9K views18 slides
Robust Large-Scale Machine Learning in the Cloud by
Robust Large-Scale Machine Learning in the CloudRobust Large-Scale Machine Learning in the Cloud
Robust Large-Scale Machine Learning in the CloudYuto Yamaguchi
1.2K views30 slides

More Related Content

More from Yuto Yamaguchi

OMNI-Prop: Seamless Node Classification on Arbitrary Label Correlation by
OMNI-Prop: Seamless Node Classification on Arbitrary Label CorrelationOMNI-Prop: Seamless Node Classification on Arbitrary Label Correlation
OMNI-Prop: Seamless Node Classification on Arbitrary Label CorrelationYuto Yamaguchi
3.5K views14 slides
Online User Location Inference Exploiting Spatiotemporal Correlations in Soci... by
Online User Location Inference Exploiting Spatiotemporal Correlations in Soci...Online User Location Inference Exploiting Spatiotemporal Correlations in Soci...
Online User Location Inference Exploiting Spatiotemporal Correlations in Soci...Yuto Yamaguchi
2.6K views30 slides
SIGMOD2013勉強会:Social Media by
SIGMOD2013勉強会:Social MediaSIGMOD2013勉強会:Social Media
SIGMOD2013勉強会:Social MediaYuto Yamaguchi
1.2K views19 slides
Towards Social User Profiling: Unified and Discriminative Influence Model for... by
Towards Social User Profiling: Unified and Discriminative Influence Model for...Towards Social User Profiling: Unified and Discriminative Influence Model for...
Towards Social User Profiling: Unified and Discriminative Influence Model for...Yuto Yamaguchi
3.3K views36 slides
The Length of Bridge Ties: Structural and Geographic Properties of Online So... by
The Length of Bridge Ties: Structural and Geographic Properties of Online So...The Length of Bridge Ties: Structural and Geographic Properties of Online So...
The Length of Bridge Ties: Structural and Geographic Properties of Online So...Yuto Yamaguchi
834 views15 slides
WWW2012勉強会:Information Diffusion in Social Networks by
WWW2012勉強会:Information Diffusion in Social NetworksWWW2012勉強会:Information Diffusion in Social Networks
WWW2012勉強会:Information Diffusion in Social NetworksYuto Yamaguchi
919 views20 slides

More from Yuto Yamaguchi(7)

OMNI-Prop: Seamless Node Classification on Arbitrary Label Correlation by Yuto Yamaguchi
OMNI-Prop: Seamless Node Classification on Arbitrary Label CorrelationOMNI-Prop: Seamless Node Classification on Arbitrary Label Correlation
OMNI-Prop: Seamless Node Classification on Arbitrary Label Correlation
Yuto Yamaguchi3.5K views
Online User Location Inference Exploiting Spatiotemporal Correlations in Soci... by Yuto Yamaguchi
Online User Location Inference Exploiting Spatiotemporal Correlations in Soci...Online User Location Inference Exploiting Spatiotemporal Correlations in Soci...
Online User Location Inference Exploiting Spatiotemporal Correlations in Soci...
Yuto Yamaguchi2.6K views
SIGMOD2013勉強会:Social Media by Yuto Yamaguchi
SIGMOD2013勉強会:Social MediaSIGMOD2013勉強会:Social Media
SIGMOD2013勉強会:Social Media
Yuto Yamaguchi1.2K views
Towards Social User Profiling: Unified and Discriminative Influence Model for... by Yuto Yamaguchi
Towards Social User Profiling: Unified and Discriminative Influence Model for...Towards Social User Profiling: Unified and Discriminative Influence Model for...
Towards Social User Profiling: Unified and Discriminative Influence Model for...
Yuto Yamaguchi3.3K views
The Length of Bridge Ties: Structural and Geographic Properties of Online So... by Yuto Yamaguchi
The Length of Bridge Ties: Structural and Geographic Properties of Online So...The Length of Bridge Ties: Structural and Geographic Properties of Online So...
The Length of Bridge Ties: Structural and Geographic Properties of Online So...
Yuto Yamaguchi834 views
WWW2012勉強会:Information Diffusion in Social Networks by Yuto Yamaguchi
WWW2012勉強会:Information Diffusion in Social NetworksWWW2012勉強会:Information Diffusion in Social Networks
WWW2012勉強会:Information Diffusion in Social Networks
Yuto Yamaguchi919 views
ICDE2012勉強会:Social Media by Yuto Yamaguchi
ICDE2012勉強会:Social MediaICDE2012勉強会:Social Media
ICDE2012勉強会:Social Media
Yuto Yamaguchi743 views

Recently uploaded

Future of Learning - Yap Aye Wee.pdf by
Future of Learning - Yap Aye Wee.pdfFuture of Learning - Yap Aye Wee.pdf
Future of Learning - Yap Aye Wee.pdfNUS-ISS
38 views11 slides
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy by
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy
"Role of a CTO in software outsourcing company", Yuriy NakonechnyyFwdays
40 views21 slides
"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi by
"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi
"AI Startup Growth from Idea to 1M ARR", Oleksandr UspenskyiFwdays
26 views9 slides
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum... by
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...NUS-ISS
28 views35 slides
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ... by
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ..."Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ...
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ...Fwdays
33 views39 slides

Recently uploaded(20)

Future of Learning - Yap Aye Wee.pdf by NUS-ISS
Future of Learning - Yap Aye Wee.pdfFuture of Learning - Yap Aye Wee.pdf
Future of Learning - Yap Aye Wee.pdf
NUS-ISS38 views
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy by Fwdays
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy
Fwdays40 views
"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi by Fwdays
"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi
"AI Startup Growth from Idea to 1M ARR", Oleksandr Uspenskyi
Fwdays26 views
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum... by NUS-ISS
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...
NUS-ISS28 views
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ... by Fwdays
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ..."Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ...
"Quality Assurance: Achieving Excellence in startup without a Dedicated QA", ...
Fwdays33 views
Transcript: The Details of Description Techniques tips and tangents on altern... by BookNet Canada
Transcript: The Details of Description Techniques tips and tangents on altern...Transcript: The Details of Description Techniques tips and tangents on altern...
Transcript: The Details of Description Techniques tips and tangents on altern...
BookNet Canada119 views
MemVerge: Past Present and Future of CXL by CXL Forum
MemVerge: Past Present and Future of CXLMemVerge: Past Present and Future of CXL
MemVerge: Past Present and Future of CXL
CXL Forum110 views
Future of Learning - Khoong Chan Meng by NUS-ISS
Future of Learning - Khoong Chan MengFuture of Learning - Khoong Chan Meng
Future of Learning - Khoong Chan Meng
NUS-ISS31 views
MemVerge: Memory Viewer Software by CXL Forum
MemVerge: Memory Viewer SoftwareMemVerge: Memory Viewer Software
MemVerge: Memory Viewer Software
CXL Forum118 views
"Fast Start to Building on AWS", Igor Ivaniuk by Fwdays
"Fast Start to Building on AWS", Igor Ivaniuk"Fast Start to Building on AWS", Igor Ivaniuk
"Fast Start to Building on AWS", Igor Ivaniuk
Fwdays36 views
Five Things You SHOULD Know About Postman by Postman
Five Things You SHOULD Know About PostmanFive Things You SHOULD Know About Postman
Five Things You SHOULD Know About Postman
Postman25 views
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur by Fwdays
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur
Fwdays40 views
Spesifikasi Lengkap ASUS Vivobook Go 14 by Dot Semarang
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14
Dot Semarang35 views
Web Dev - 1 PPT.pdf by gdsczhcet
Web Dev - 1 PPT.pdfWeb Dev - 1 PPT.pdf
Web Dev - 1 PPT.pdf
gdsczhcet52 views
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa... by The Digital Insurer
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...
TE Connectivity: Card Edge Interconnects by CXL Forum
TE Connectivity: Card Edge InterconnectsTE Connectivity: Card Edge Interconnects
TE Connectivity: Card Edge Interconnects
CXL Forum96 views
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad... by Fwdays
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad..."Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad...
"Ukrainian Mobile Banking Scaling in Practice. From 0 to 100 and beyond", Vad...
Fwdays40 views
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV by Splunk
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
Splunk86 views

When Does Label Propagation Fail? A View from a Network Generative Model

  • 1. When  Does  Label  Propaga1on  Fail?   A  View  from  a  Network  Genera1ve  Model Yuto  Yamaguchi  and  Kohei  Hayashi 17/08/22 IJCAI@Melbourne 1
  • 2. Node Classification Given Find Partially labeled undirected graph Labels of all nodes 17/08/22 IJCAI@Melbourne 2
  • 3. Example: User profile inference Friends Soccer Soccer Soccer Tennis Baseball ??? What’s his hobby? Node Classification 17/08/22 IJCAI@Melbourne 3
  • 4. Label Propagation (1/2) Propagate neighbors’ labels Friends Soccer Soccer Soccer Tennis Baseline ??? Soccer Soccer Soccer Tennis Baseline Soccer [Zhu+, 03], [Zhou+, 03], etc. 17/08/22 IJCAI@Melbourne 4
  • 5. Label Propagation (2/2) Q F;X,Y,λ( )= 1 2 fi − yi 2 2 i=1 N ∑ + λ 2 xij fi − fj 2 2 j=1 N ∑ i=1 N ∑ Given: adjacency matrix X and labels Y Find: F = { fi } that minimizes Q 17/08/22 IJCAI@Melbourne 5 F ∈ RN x K Y ∈ {0, 1}N x K X ∈ {0, 1}N x N N: # of nodes K: # of labels λ ∈ R+ : user parameter [Zhu+, 03], [Zhou+, 03], etc.
  • 6. Cases  when  LP  fails  (prac1cally  known) Different labels are connected Label ratio is not uniform Q. So, do we know why LP fails in these cases? A. No. Since it’s not a probabilistic model, we don’t know the assumptions behind the model. 17/08/22 IJCAI@Melbourne 6 Edge probability is not uniform
  • 7. What  we  do  in  this  work 1.  Prove  a  theore1cal  rela1onship  between  LP   and  Stochas(c  Block  Model,  which  is  a  well-­‐ studied  probabilis1c  genera1ve  model   2.  Find  the  assump(ons  behind  LP  through  the   assump1ons  behind  SBM   3.  Show  when  and  why  LP  fails 17/08/22 IJCAI@Melbourne 7
  • 9. Stochastic Block Model Generative process Multinomial Bernoulli ① ② ①: Generate cluster assignment for each node (which can be thought of labels) ②: Generate adjacency matrix 17/08/22 IJCAI@Melbourne 9 γ ∈ RK Π ∈ RKxK Parameters:
  • 10. Proposed: Partially Labeled SBM (PLSBM) Generative process ① ② ③ ②:Generate labels for “labeled nodes” (α large à yi is more likely to be the same as zi) Depends on parameter α 17/08/22 IJCAI@Melbourne 10 γ ∈ RK Π ∈ RKxK α ∈ 0,1[ ] Parameters:
  • 11. Rela1onships  between  models 17/08/22 IJCAI@Melbourne 11 SBM PLSBM LP Discre1zed  LP Main  result   (next  slide) No  labels Con1nuous   relaxa1on
  • 12. Main Result Map estimator Z of PLSBM is identical to the solution of (discretized) LP when the following conditions hold Condition 1: Condition 2: Condition 3: Condition 4: (omitted) 17/08/22 IJCAI@Melbourne 12
  • 13. Condition 1 Implication (implicit assumption of LP) •  Label ratio is uniform 17/08/22 IJCAI@Melbourne 13 Violates this assumption L
  • 14. Condition 2 Implication (Implicit assumptions of LP) •  Edge probs between the same labels are all the same (μ) •  Edge probs between different labels are all the same (ν) 17/08/22 IJCAI@Melbourne 14 Violates this assumption L
  • 15. Condition 3 Implication (Implicit assumption of LP) •  Assortative (same labels tend to be connected) 17/08/22 IJCAI@Melbourne 15 Violates this assumption L
  • 16. Experimental results 17/08/22 IJCAI@Melbourne 16 … Come see full results at the poster session J Better Setups: 1.  Generate datasets by PLSBM 2.  infer labels (Z) by PLSBM, SBM, and LP 3.  Report mean accuracy of 20 trials Assortative Disassortative Agree with theoretical results
  • 17. Summary •  Proposed  Par1ally-­‐Labeled  SBM  (PLSBM)   •  Proved  the  rela1onship  between  LP  and  SBM  via   PLSBM   •  Showed  cases  when  LP  fails   •  Experimental  and  Theore1cal  results  agree 17/08/22 IJCAI@Melbourne 17 Github: yamaguchiyuto/plsbm