SlideShare a Scribd company logo

Limits of community detection

•
•
Vincent Traag
Vincent Traag

This document discusses two key problems with modularity-based community detection in networks: incomparability and resolution limit. The modularity measure cannot reliably distinguish between networks with genuine communities and networks with no communities. Additionally, the size of communities detected depends on the overall network size, so smaller true communities may not be detected in large networks. The document provides examples of networks where modularity fails to identify the true partition into communities.

Science
1 of 52
Download to read offline
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Limits of community detection V.A. Traag1, P. Van Dooren1, Y.E. Nesterov2 1ICTEAM Universit´e Catholique de Louvain 2CORE Universit´e Catholique de Louvain 20 March 2012
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Understanding Networks Facebook (2011)
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Understanding Networks Facebook (2011)
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Understanding Networks Blogosphere Presidential Election (2004)
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Understanding Networks Aad Kosto Ab Klink Abraham Kuyper Abu Nuwas Ad Melkert Adri Duivesteijn AEL Afshin Ellian Ahmed Aboutaleb Ahmet Daskapan Aisha AIVD Ali B. Ali Lazrak Ali Madanipour Al-Qaeda Amin Saikal Amitai Etzioni Anil Ramdas Anita Bocker Annelies Verstand Anne-Ruth Wertheim Anton Zijderveld Arie Slob Arie van der Zwan Arie van Deursen Aristoteles Averroes Avicenna Avishai Margalit Ayaan Hirsi Ali Ayatolah Khomeini Bart Jan Spruyt Bart Tromp Bas Heijne Bas van Stokkom Bernard Lewis Bert Middel Bertrand Russell Bert Wagendorp Betsy Udink Osama Bin Laden Britta Bohler Camiel Eurlings CDA CIDI Claude Lefort Claude Steele Commissie Blok Commissie-Stasi Commissie van Montfrans CPBD66 Daniel Garrison Brinton David Brooks David Lilienthal David Pryce-Jones De Volkskrant Dick Cheney Dick de Ruijter Dick Pels Dick van Eijk Dietrich Thranhardt Driss el Boujoufi Dyab Abou Jahjah Ebru Umar Ed Leuw Edmund Burke Edward Said Een Ander Joods Geluid Ella Kalsbeek Elsbeth Etty Els Borst Erasmus Erica Terpstra Erik Snel Ernest Renan Fadima Orgu Fatima Elatik Fatma Katirci Federatie Nederlandse Zionisten Femke Halsema FNV Fokke Obbema Forum Freek de Jonge Frits Bolkestein Frits van veen Frits WesterFuad Hussein Gabriel van den Brink Geert Mak Geert Wilders George W. Bush Gerard de Vries Gerard Smink Gerard Spong Gerrit Zalm Gijs Weenink Gilles Kepel Groen Links Haci Karacaer Hafid Bouazza Halim El Madkouri Han Entzinger Hans Boutellier Hans Dijkstal Hans Janmaat Hans Siebers Hans Visser Harry van Doorn Hendrik Colijn Henk Doll Herbert Gans Herman Philipse Herman Vuijsje Hilbrand Nawijn H.J.A. Hofland Hoge Commissariaat voor Vluchtelingen Hugo Brandt Corstius Human Development Report Ian Buruma Ibn Hazam Ibn Warraq Ilhan Akel imam El-Moumni Immanuel Kant Irshad Manji Ivo Opstelten Jacob Kohnstamm Jaco Dagevos Jacqueline Costa-Lascoux Jacqueline Draaijer Jacques Wallage James C. Kennedy Jan Beerenhout Jan Blokker Jan BrugmanJan Drentje Jan Jacob Slauerhoff Jan-Peter Balkendende Jan Pronk Jan Rijpstra Jan Schaefer Jantine Nipius Jantine Oldersma Jan Willem Duyvendak Jean Tillie Jefferson Jelle van der Meer Jerome Heldring Job Cohen Johan Norberg Johan Remkes John Coetzee John Gray John Jansen van galen John Leerdam John Mollenkopf John Stuart Mill Jola Jakson Joodse Gemeente Amsterdam Joods Journaal Joost Eerdmans Joost Niemoller Joost Zwagerman Jorg Haider Jos de Beus Jos de Mul Jozias van Aartsen Justus Veenman Jytte Klausen Kader Abdollah Karel van het Reve Karen Adelmund Kees van der Staaij Kees van Kooten Kees Vendrik Khalifa KMAN Kohnstamm Instituut Kohnstamm-rapport Laurent Chambon Leefbaar Rotterdam Leo Lucassen Likud Nederland LPF Maarten Huygen Malcolm X Mansour Khalid Marc de Kessel Marcel van Dam Marco Borsato Marco Pastors Maria van der Hoeven Marijke Vos Marion van San Mark Bovens Marlite Halbertsma Martien Kromwijk Martin Luther Maududi Maurits Berger Maxime Verhagen Meindert Fennema Menno Hurenkamp Micha de Winter Michele de Waard Michele Tribalat Milli Gorus Mimount Bousakla Mirjam de Rijk Mohammed Arkoun Mohammed Benzakour Mohammed Cheppih Mohammed Iqbal Mohammed Reza Pahlavi Mohammed Sini Naema Tahir Naima Elmaslouhi Nazih Ayubi Nazmiye Oral NCB Neal Ascherson Nebahat Albayrak Nico de Haas Nico van Nimwegen Nieuw Israelitisch Weekblad Norman Podhoretz Nout Wellink NRC Handelsblad Olivier Roy Orhan Pamuk Oscar Garschagen Oscar Hammerstein Osdorp Posse Paul Cliteur Paul de Beer Paul Gerbrands Pauline Meurs Paul Meurs Paul Rosenmuller Paul Scheffer Paul Schnabel Peter Langendam Peter Smit Pierre Bourdieu Pierre Heijnen Pieter Lakeman Piet Hein Donner Pim Fortuyn PPR President Eisenhower Prinses Maxima Profeet Mohammed Professor Peters PvdA Qotb Ramsey Nasr Renate Rubinstein Rene Cuperus RIAGG Ria van Gils Rita Verdonk Rob Oudkerk Roger van Boxtel Ronald van Raak Rudy Kousbroek Ruud Koopmans Ruud Lubbers Saddam Hussein Said Benayad Salman Rushdie Samuel Huntington Schelto Patijn SCP Senay Ozdemir Shervin Nekuee Shukri Silvio Berlusconi Simone van der Burg SISWO Sjaak van der Tak Sjeik Ahmed Yassin Sjoerd de Jong Soedish Verhoeven SP Spinoza Stef Blok Susan Moller Okin S.W. Couwenberg Sylvain Ephimenco Tara Singh Varma Theodor Adorno Theo van Gogh Theo Veenkamp Thijs WoltgensThorbecke Timothy Garton Ash Tiny Cox Tony Blair Tzvetan Todorov Van Montfrans Vlaams Blok VN Committee for elimination of all forms of racism Volkert van der Graaf VVD VVN Wasif Shadid Werner Schiffauer Wiardi Beckman Stichting Willem Vermeend Wim de Bie Wim Kok Wim Willems Wouter bos WRR WRR 1989 WRR 2001 WRR 2003 Yucel Yesilgoz Zekeriya Gumus Zeki Arslan Dutch Debate on Integration (2002-04),
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Community Detection • Detect ‘natural’ communities in networks • Basic idea: ‘relatively’ many links inside communities
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free First principle approach • Reward “good” links, penalize “bad” links. • Assume contribution for links within and between are equal. • Simplifies to only internal links. Present Missing Within + − Between − + Link present or missing? Link within or between community? Reward (+) or Penalize (−)
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free First principle approach • Reward “good” links, penalize “bad” links. • Assume contribution for links within and between are equal. • Simplifies to only internal links. Aij = 1 Aij = 0 δij = 1 aij −bij δij = 0 −cij dij Link present or missing? Link within or between community? Reward (+) or Penalize (−)
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free First principle approach • Reward “good” links, penalize “bad” links. • Assume contribution for links within and between are equal. • Simplifies to only internal links. Aij = 1 Aij = 0 δij = 1 aij −bij δij = 0 −aij bij Link present or missing? Link within or between community? Reward (+) or Penalize (−)
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free First principle approach • Reward “good” links, penalize “bad” links. • Assume contribution for links within and between are equal. • Simplifies to only internal links. Objective function H = − ij (aij Aij −bij (1 − Aij )) δij
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Null-model Objective function H = − ij (aij Aij −bij (1 − Aij )) δij • Introduce weights aij = 1 − γRBpij and bij = γRBpij . • Null-model pij , constrained by ij pij = 2m. • Parameter γ known as resolution parameter: Higher γ ⇒ smaller communities. Lower γ ⇒ larger communities. • Leads to HRB = − ij (Aij − γpij ) δij .
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Configuration Null-model Reichard-Bornholdt objective function HRB = − ij (Aij − γpij ) δij • Rewire links, but keep degrees unchanged • Probability link between i and j is then pij = ki kj 2m . i jki kj
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Configuration Null-model Reichard-Bornholdt objective function HRB = − ij (Aij − γpij ) δij • Rewire links, but keep degrees unchanged • Probability link between i and j is then pij = ki kj 2m . i jki kj γ = 1 leads to modularity Q = 1 2m ij Aij − ki kj 2m δij ∼ −H.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Basic properties Modularity Q = 1 2m ij Aij − ki kj 2m δij • Normalized: between −1 (bad partition) and 1 (good partition). • Trivial partitions have modularity Q = 0. • No community consists of a single node. • Each community is internally connected. • Modularity maximization is NP-complete.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Examples: grid Modularity tends to 1.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Examples: tree Modularity tends to 1.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Examples: random graph Random partition has Q ∼ 0, but best partition has Q ∼ √ k k > 0.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Problem 1: Incomparability Modularity can be high even “without communities”
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Compare modularity scores (Dis)agreement on whether smoking is cancerous.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Scientific specialization
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Scientific specialization
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Scientific specialization
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: ring of cliques Modularity might merge cliques
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: ring of rings Modularity might split rings
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: Cliques of different size No longer fix γ = 1: tune γ so that it finds “correct” partition. No γ yields “correct” partition.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: broad community sizes Competing constraints: • Lower γ ⇒ merge smaller cliques. • Higher γ ⇒ split large cliques. No γ yields “correct” partition.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: broad community sizes Competing constraints: • Lower γ ⇒ merge smaller cliques. • Higher γ ⇒ split large cliques. No γ yields “correct” partition.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: broad community sizes Competing constraints: • Lower γ ⇒ merge smaller cliques. • Higher γ ⇒ split large cliques. No γ yields “correct” partition.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Problem 2: Resolution-limit Size of communities might depend on size of network
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Resolution limit revisited Resolution-limit Resolution-limit-free • Problem is not merging per s´e. • Rather, cliques separate in subgraph, but merge in large graph (or vice versa).
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Resolution limit revisited Resolution-limit Resolution-limit-free Definition (Resolution-limit-free) Objective function H is called resolution-limit-free if, whenever partition C optimal for G, then subpartition D ⊂ C also optimal for subgraph H(D) ⊂ G induced by D.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Resolution-limit methods General community detection H = − ij (aij Aij −bij (1 − Aij )) δij Not resolution-limit-free RB model Set aij = 1 − γpij , bij = γRBpij . Modularity Set pij = ki kj 2m and γ = 1 What weights aij and bij to choose so that model is resolution-limit-free?
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Resolution-limit-free methods General community detection H = − ij (aij Aij −bij (1 − Aij )) δij Resolution-limit-free RN model Set aij = 1, bij = γRN. CPM Set aij = 1 − γ and bij = γ. Leads to H = − ij (Aij − γ)δij . Are there any other weights aij and bij ?
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Local weights General community detection H = − ij (aij Aij −bij (1 − Aij )) δij i j Definition (Local weights) Weights local when they only depend on subgraph induced by i and j.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Local weights General community detection H = − ij (aij Aij −bij (1 − Aij )) δij i j Theorem (Local weights ⇒ resolution-limit-free) Method is resolution-limit-free if weights are local.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Local weights General community detection H = − ij (aij Aij −bij (1 − Aij )) δij i j Are local weights necessary?
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free “Almost” resolution-free ⇒ local weights α1 α2 α3 β1 β2 β3 β4 Not resolution free, whenever merged in large graph, but separate in subgraph, or the other way around. So Hm < Hs and Hm > Hs, or Hm > Hs and Hm < Hs
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free “Almost” resolution-free ⇒ local weights α1 α2 α3 β1 β2 β3 β4 Working out, yields α3(β4(nc − 1) + 2β3) < α3(β4(nc − 1) + 2β3) or, α3(β4(nc − 1) + 2β3) > α3(β4(nc − 1) + 2β3). Always satisfied for non-local weights, except when (nc − 1) = 2 β3α3 − β3α3 β4α3 − β4α3 .
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 57 5 6 2 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Performance (directed networks) µ0 0.2 0.4 0.6 0.8 1.0 NMI 0.25 0.5 0.75 1 CPM γ=γ∗ ER γ=p RB Conf γRB=γ∗ RB Mod. γRB=1 Inf. n = 103 n = 104
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Problems of CPM No problems: • Doesn’t merge cliques. • Doesn’t split rings. • Correctly detects cliques of different size. Remaining problem: • Communities of different density.
Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Conclusions • Modularity high even when no communities. • Modularity merge/splits communities unexpectedly (resolution-limit). • Methods using local weights are resolution-limit-free. • Resolution-limit-free method performs superbly. Open question: • what when communities have different densities? Thank you for your attention. Questions?

Recommended

Khaled wahba msm
Khaled wahba msmKhaled wahba msm
Khaled wahba msm
RevistaBiz
 
Dynamic Community Detection for Large-scale e-Commerce data with Spark Stream...
Dynamic Community Detection for Large-scale e-Commerce data with Spark Stream...Dynamic Community Detection for Large-scale e-Commerce data with Spark Stream...
Dynamic Community Detection for Large-scale e-Commerce data with Spark Stream...
Spark Summit
 
GraphFrames: DataFrame-based graphs for Apache® Spark™
GraphFrames: DataFrame-based graphs for Apache® Spark™GraphFrames: DataFrame-based graphs for Apache® Spark™
GraphFrames: DataFrame-based graphs for Apache® Spark™
Databricks
 
GraphFrames: Graph Queries in Spark SQL by Ankur Dave
GraphFrames: Graph Queries in Spark SQL by Ankur DaveGraphFrames: Graph Queries in Spark SQL by Ankur Dave
GraphFrames: Graph Queries in Spark SQL by Ankur Dave
Spark Summit
 
Peer review uncertainty at the institutional level
Peer review uncertainty at the institutional levelPeer review uncertainty at the institutional level
Peer review uncertainty at the institutional level
Vincent Traag
 
Replacing peer review by metrics in the UK REF?
Replacing peer review by metrics in the UK REF?Replacing peer review by metrics in the UK REF?
Replacing peer review by metrics in the UK REF?
Vincent Traag
 
Use of the journal impact factor for assessing individual articles need not b...
Use of the journal impact factor for assessing individual articles need not b...Use of the journal impact factor for assessing individual articles need not b...
Use of the journal impact factor for assessing individual articles need not b...
Vincent Traag
 
Uncovering important intermediate publications
Uncovering important intermediate publicationsUncovering important intermediate publications
Uncovering important intermediate publications
Vincent Traag
 

Recommended

Khaled wahba msm
Khaled wahba msmKhaled wahba msm
Khaled wahba msm
RevistaBiz
 
Dynamic Community Detection for Large-scale e-Commerce data with Spark Stream...
Dynamic Community Detection for Large-scale e-Commerce data with Spark Stream...Dynamic Community Detection for Large-scale e-Commerce data with Spark Stream...
Dynamic Community Detection for Large-scale e-Commerce data with Spark Stream...
Spark Summit
 
GraphFrames: DataFrame-based graphs for Apache® Spark™
GraphFrames: DataFrame-based graphs for Apache® Spark™GraphFrames: DataFrame-based graphs for Apache® Spark™
GraphFrames: DataFrame-based graphs for Apache® Spark™
Databricks
 
GraphFrames: Graph Queries in Spark SQL by Ankur Dave
GraphFrames: Graph Queries in Spark SQL by Ankur DaveGraphFrames: Graph Queries in Spark SQL by Ankur Dave
GraphFrames: Graph Queries in Spark SQL by Ankur Dave
Spark Summit
 
Peer review uncertainty at the institutional level
Peer review uncertainty at the institutional levelPeer review uncertainty at the institutional level
Peer review uncertainty at the institutional level
Vincent Traag
 
Replacing peer review by metrics in the UK REF?
Replacing peer review by metrics in the UK REF?Replacing peer review by metrics in the UK REF?
Replacing peer review by metrics in the UK REF?
Vincent Traag
 
Use of the journal impact factor for assessing individual articles need not b...
Use of the journal impact factor for assessing individual articles need not b...Use of the journal impact factor for assessing individual articles need not b...
Use of the journal impact factor for assessing individual articles need not b...
Vincent Traag
 
Uncovering important intermediate publications
Uncovering important intermediate publicationsUncovering important intermediate publications
Uncovering important intermediate publications
Vincent Traag
 
Complex contagion of campaign donations
Complex contagion of campaign donationsComplex contagion of campaign donations
Complex contagion of campaign donations
Vincent Traag
 
Polarization and consensus in citation networks
Polarization and consensus in citation networksPolarization and consensus in citation networks
Polarization and consensus in citation networks
Vincent Traag
 
Community structure in complex networks
Community structure in complex networksCommunity structure in complex networks
Community structure in complex networks
Vincent Traag
 
Introduction to complex networks
Introduction to complex networksIntroduction to complex networks
Introduction to complex networks
Vincent Traag
 
Public thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networksPublic thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networks
Vincent Traag
 
Structure of media attention
Structure of media attentionStructure of media attention
Structure of media attention
Vincent Traag
 
Dynamics of Media Attention
Dynamics of Media AttentionDynamics of Media Attention
Dynamics of Media Attention
Vincent Traag
 
Dynamical Models Explaining Social Balance
Dynamical Models Explaining Social BalanceDynamical Models Explaining Social Balance
Dynamical Models Explaining Social Balance
Vincent Traag
 
Significant scales in community structure
Significant scales in community structureSignificant scales in community structure
Significant scales in community structure
Vincent Traag
 
Reconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from NewspapersReconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from Newspapers
Vincent Traag
 
Reputation Dynamics Through Gossiping
Reputation Dynamics Through GossipingReputation Dynamics Through Gossiping
Reputation Dynamics Through Gossiping
Vincent Traag
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
Vincent Traag
 
Resolution-free community detection
Resolution-free community detectionResolution-free community detection
Resolution-free community detection
Vincent Traag
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
Vincent Traag
 
Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.
Vincent Traag
 
Social Event Detection
Social Event DetectionSocial Event Detection
Social Event Detection
Vincent Traag
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
Vincent Traag
 
Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.
Vincent Traag
 
Cooperation and Reputation
Cooperation and ReputationCooperation and Reputation
Cooperation and Reputation
Vincent Traag
 
Slice modularity
Slice modularitySlice modularity
Slice modularity
Vincent Traag
 
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATIONPRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
ChetanK57
 
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Sérgio Sacani
 

More Related Content

More from Vincent Traag

Complex contagion of campaign donations
Complex contagion of campaign donationsComplex contagion of campaign donations
Complex contagion of campaign donations
Vincent Traag
 
Polarization and consensus in citation networks
Polarization and consensus in citation networksPolarization and consensus in citation networks
Polarization and consensus in citation networks
Vincent Traag
 
Community structure in complex networks
Community structure in complex networksCommunity structure in complex networks
Community structure in complex networks
Vincent Traag
 
Introduction to complex networks
Introduction to complex networksIntroduction to complex networks
Introduction to complex networks
Vincent Traag
 
Public thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networksPublic thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networks
Vincent Traag
 
Structure of media attention
Structure of media attentionStructure of media attention
Structure of media attention
Vincent Traag
 
Dynamics of Media Attention
Dynamics of Media AttentionDynamics of Media Attention
Dynamics of Media Attention
Vincent Traag
 
Dynamical Models Explaining Social Balance
Dynamical Models Explaining Social BalanceDynamical Models Explaining Social Balance
Dynamical Models Explaining Social Balance
Vincent Traag
 
Significant scales in community structure
Significant scales in community structureSignificant scales in community structure
Significant scales in community structure
Vincent Traag
 
Reconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from NewspapersReconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from Newspapers
Vincent Traag
 
Reputation Dynamics Through Gossiping
Reputation Dynamics Through GossipingReputation Dynamics Through Gossiping
Reputation Dynamics Through Gossiping
Vincent Traag
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
Vincent Traag
 
Resolution-free community detection
Resolution-free community detectionResolution-free community detection
Resolution-free community detection
Vincent Traag
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
Vincent Traag
 
Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.
Vincent Traag
 
Social Event Detection
Social Event DetectionSocial Event Detection
Social Event Detection
Vincent Traag
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
Vincent Traag
 
Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.
Vincent Traag
 
Cooperation and Reputation
Cooperation and ReputationCooperation and Reputation
Cooperation and Reputation
Vincent Traag
 
Slice modularity
Slice modularitySlice modularity
Slice modularity
Vincent Traag
 

More from Vincent Traag (20)

Complex contagion of campaign donations
Complex contagion of campaign donationsComplex contagion of campaign donations
Complex contagion of campaign donations
 
Polarization and consensus in citation networks
Polarization and consensus in citation networksPolarization and consensus in citation networks
Polarization and consensus in citation networks
 
Community structure in complex networks
Community structure in complex networksCommunity structure in complex networks
Community structure in complex networks
 
Introduction to complex networks
Introduction to complex networksIntroduction to complex networks
Introduction to complex networks
 
Public thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networksPublic thesis defence: groups and reputation in social networks
Public thesis defence: groups and reputation in social networks
 
Structure of media attention
Structure of media attentionStructure of media attention
Structure of media attention
 
Dynamics of Media Attention
Dynamics of Media AttentionDynamics of Media Attention
Dynamics of Media Attention
 
Dynamical Models Explaining Social Balance
Dynamical Models Explaining Social BalanceDynamical Models Explaining Social Balance
Dynamical Models Explaining Social Balance
 
Significant scales in community structure
Significant scales in community structureSignificant scales in community structure
Significant scales in community structure
 
Reconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from NewspapersReconstructing Third World Elite Rotation Events from Newspapers
Reconstructing Third World Elite Rotation Events from Newspapers
 
Reputation Dynamics Through Gossiping
Reputation Dynamics Through GossipingReputation Dynamics Through Gossiping
Reputation Dynamics Through Gossiping
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
 
Resolution-free community detection
Resolution-free community detectionResolution-free community detection
Resolution-free community detection
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
 
Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.
 
Social Event Detection
Social Event DetectionSocial Event Detection
Social Event Detection
 
Cooperation, Reputation & Gossiping
Cooperation, Reputation & GossipingCooperation, Reputation & Gossiping
Cooperation, Reputation & Gossiping
 
Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.Exponential Ranking: Taking into account negative links.
Exponential Ranking: Taking into account negative links.
 
Cooperation and Reputation
Cooperation and ReputationCooperation and Reputation
Cooperation and Reputation
 
Slice modularity
Slice modularitySlice modularity
Slice modularity
 

Recently uploaded

PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATIONPRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
ChetanK57
 
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Sérgio Sacani
 
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
yqqaatn0
 
GBSN - Microbiology (Lab 4) Culture Media
GBSN - Microbiology (Lab 4) Culture MediaGBSN - Microbiology (Lab 4) Culture Media
GBSN - Microbiology (Lab 4) Culture Media
Areesha Ahmad
 
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...
University of Maribor
 
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Sérgio Sacani
 
DERIVATION OF MODIFIED BERNOULLI EQUATION WITH VISCOUS EFFECTS AND TERMINAL V...
DERIVATION OF MODIFIED BERNOULLI EQUATION WITH VISCOUS EFFECTS AND TERMINAL V...DERIVATION OF MODIFIED BERNOULLI EQUATION WITH VISCOUS EFFECTS AND TERMINAL V...
DERIVATION OF MODIFIED BERNOULLI EQUATION WITH VISCOUS EFFECTS AND TERMINAL V...
Wasswaderrick3
 
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
University of Maribor
 
Orion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWSOrion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWS
Columbia Weather Systems
 
BLOOD AND BLOOD COMPONENT- introduction to blood physiology
BLOOD AND BLOOD COMPONENT- introduction to blood physiologyBLOOD AND BLOOD COMPONENT- introduction to blood physiology
BLOOD AND BLOOD COMPONENT- introduction to blood physiology
NoelManyise1
 
Introduction to Mean Field Theory(MFT).pptx
Introduction to Mean Field Theory(MFT).pptxIntroduction to Mean Field Theory(MFT).pptx
Introduction to Mean Field Theory(MFT).pptx
zeex60
 
Lateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensiveLateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensive
silvermistyshot
 
Mudde & Rovira Kaltwasser. - Populism - a very short introduction [2017].pdf
Mudde & Rovira Kaltwasser. - Populism - a very short introduction [2017].pdfMudde & Rovira Kaltwasser. - Populism - a very short introduction [2017].pdf
Mudde & Rovira Kaltwasser. - Populism - a very short introduction [2017].pdf
frank0071
 
DMARDs Pharmacolgy Pharm D 5th Semester.pdf
DMARDs Pharmacolgy Pharm D 5th Semester.pdfDMARDs Pharmacolgy Pharm D 5th Semester.pdf
DMARDs Pharmacolgy Pharm D 5th Semester.pdf
fafyfskhan251kmf
 
Chapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisisChapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisis
tonzsalvador2222
 
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Ana Luísa Pinho
 
Unveiling the Energy Potential of Marshmallow Deposits.pdf
Unveiling the Energy Potential of Marshmallow Deposits.pdfUnveiling the Energy Potential of Marshmallow Deposits.pdf
Unveiling the Energy Potential of Marshmallow Deposits.pdf
Erdal Coalmaker
 
Richard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlandsRichard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlands
Richard Gill
 
3D Hybrid PIC simulation of the plasma expansion (ISSS-14)
3D Hybrid PIC simulation of the plasma expansion (ISSS-14)3D Hybrid PIC simulation of the plasma expansion (ISSS-14)
3D Hybrid PIC simulation of the plasma expansion (ISSS-14)
David Osipyan
 
role of pramana in research.pptx in science
role of pramana in research.pptx in sciencerole of pramana in research.pptx in science
role of pramana in research.pptx in science
sonaliswain16
 

Recently uploaded (20)

PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATIONPRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
 
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
 
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
 
GBSN - Microbiology (Lab 4) Culture Media
GBSN - Microbiology (Lab 4) Culture MediaGBSN - Microbiology (Lab 4) Culture Media
GBSN - Microbiology (Lab 4) Culture Media
 
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...
 
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
 
DERIVATION OF MODIFIED BERNOULLI EQUATION WITH VISCOUS EFFECTS AND TERMINAL V...
DERIVATION OF MODIFIED BERNOULLI EQUATION WITH VISCOUS EFFECTS AND TERMINAL V...DERIVATION OF MODIFIED BERNOULLI EQUATION WITH VISCOUS EFFECTS AND TERMINAL V...
DERIVATION OF MODIFIED BERNOULLI EQUATION WITH VISCOUS EFFECTS AND TERMINAL V...
 
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
 
Orion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWSOrion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWS
 
BLOOD AND BLOOD COMPONENT- introduction to blood physiology
BLOOD AND BLOOD COMPONENT- introduction to blood physiologyBLOOD AND BLOOD COMPONENT- introduction to blood physiology
BLOOD AND BLOOD COMPONENT- introduction to blood physiology
 
Introduction to Mean Field Theory(MFT).pptx
Introduction to Mean Field Theory(MFT).pptxIntroduction to Mean Field Theory(MFT).pptx
Introduction to Mean Field Theory(MFT).pptx
 
Lateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensiveLateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensive
 
Mudde & Rovira Kaltwasser. - Populism - a very short introduction [2017].pdf
Mudde & Rovira Kaltwasser. - Populism - a very short introduction [2017].pdfMudde & Rovira Kaltwasser. - Populism - a very short introduction [2017].pdf
Mudde & Rovira Kaltwasser. - Populism - a very short introduction [2017].pdf
 
DMARDs Pharmacolgy Pharm D 5th Semester.pdf
DMARDs Pharmacolgy Pharm D 5th Semester.pdfDMARDs Pharmacolgy Pharm D 5th Semester.pdf
DMARDs Pharmacolgy Pharm D 5th Semester.pdf
 
Chapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisisChapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisis
 
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
 
Unveiling the Energy Potential of Marshmallow Deposits.pdf
Unveiling the Energy Potential of Marshmallow Deposits.pdfUnveiling the Energy Potential of Marshmallow Deposits.pdf
Unveiling the Energy Potential of Marshmallow Deposits.pdf
 
Richard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlandsRichard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlands
 
3D Hybrid PIC simulation of the plasma expansion (ISSS-14)
3D Hybrid PIC simulation of the plasma expansion (ISSS-14)3D Hybrid PIC simulation of the plasma expansion (ISSS-14)
3D Hybrid PIC simulation of the plasma expansion (ISSS-14)
 
role of pramana in research.pptx in science
role of pramana in research.pptx in sciencerole of pramana in research.pptx in science
role of pramana in research.pptx in science
 

Limits of community detection

  • 1. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Limits of community detection V.A. Traag1, P. Van Dooren1, Y.E. Nesterov2 1ICTEAM Universit´e Catholique de Louvain 2CORE Universit´e Catholique de Louvain 20 March 2012
  • 2. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
  • 3. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
  • 4. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Understanding Networks Facebook (2011)
  • 5. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Understanding Networks Facebook (2011)
  • 6. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Understanding Networks Blogosphere Presidential Election (2004)
  • 7. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Understanding Networks Aad Kosto Ab Klink Abraham Kuyper Abu Nuwas Ad Melkert Adri Duivesteijn AEL Afshin Ellian Ahmed Aboutaleb Ahmet Daskapan Aisha AIVD Ali B. Ali Lazrak Ali Madanipour Al-Qaeda Amin Saikal Amitai Etzioni Anil Ramdas Anita Bocker Annelies Verstand Anne-Ruth Wertheim Anton Zijderveld Arie Slob Arie van der Zwan Arie van Deursen Aristoteles Averroes Avicenna Avishai Margalit Ayaan Hirsi Ali Ayatolah Khomeini Bart Jan Spruyt Bart Tromp Bas Heijne Bas van Stokkom Bernard Lewis Bert Middel Bertrand Russell Bert Wagendorp Betsy Udink Osama Bin Laden Britta Bohler Camiel Eurlings CDA CIDI Claude Lefort Claude Steele Commissie Blok Commissie-Stasi Commissie van Montfrans CPBD66 Daniel Garrison Brinton David Brooks David Lilienthal David Pryce-Jones De Volkskrant Dick Cheney Dick de Ruijter Dick Pels Dick van Eijk Dietrich Thranhardt Driss el Boujoufi Dyab Abou Jahjah Ebru Umar Ed Leuw Edmund Burke Edward Said Een Ander Joods Geluid Ella Kalsbeek Elsbeth Etty Els Borst Erasmus Erica Terpstra Erik Snel Ernest Renan Fadima Orgu Fatima Elatik Fatma Katirci Federatie Nederlandse Zionisten Femke Halsema FNV Fokke Obbema Forum Freek de Jonge Frits Bolkestein Frits van veen Frits WesterFuad Hussein Gabriel van den Brink Geert Mak Geert Wilders George W. Bush Gerard de Vries Gerard Smink Gerard Spong Gerrit Zalm Gijs Weenink Gilles Kepel Groen Links Haci Karacaer Hafid Bouazza Halim El Madkouri Han Entzinger Hans Boutellier Hans Dijkstal Hans Janmaat Hans Siebers Hans Visser Harry van Doorn Hendrik Colijn Henk Doll Herbert Gans Herman Philipse Herman Vuijsje Hilbrand Nawijn H.J.A. Hofland Hoge Commissariaat voor Vluchtelingen Hugo Brandt Corstius Human Development Report Ian Buruma Ibn Hazam Ibn Warraq Ilhan Akel imam El-Moumni Immanuel Kant Irshad Manji Ivo Opstelten Jacob Kohnstamm Jaco Dagevos Jacqueline Costa-Lascoux Jacqueline Draaijer Jacques Wallage James C. Kennedy Jan Beerenhout Jan Blokker Jan BrugmanJan Drentje Jan Jacob Slauerhoff Jan-Peter Balkendende Jan Pronk Jan Rijpstra Jan Schaefer Jantine Nipius Jantine Oldersma Jan Willem Duyvendak Jean Tillie Jefferson Jelle van der Meer Jerome Heldring Job Cohen Johan Norberg Johan Remkes John Coetzee John Gray John Jansen van galen John Leerdam John Mollenkopf John Stuart Mill Jola Jakson Joodse Gemeente Amsterdam Joods Journaal Joost Eerdmans Joost Niemoller Joost Zwagerman Jorg Haider Jos de Beus Jos de Mul Jozias van Aartsen Justus Veenman Jytte Klausen Kader Abdollah Karel van het Reve Karen Adelmund Kees van der Staaij Kees van Kooten Kees Vendrik Khalifa KMAN Kohnstamm Instituut Kohnstamm-rapport Laurent Chambon Leefbaar Rotterdam Leo Lucassen Likud Nederland LPF Maarten Huygen Malcolm X Mansour Khalid Marc de Kessel Marcel van Dam Marco Borsato Marco Pastors Maria van der Hoeven Marijke Vos Marion van San Mark Bovens Marlite Halbertsma Martien Kromwijk Martin Luther Maududi Maurits Berger Maxime Verhagen Meindert Fennema Menno Hurenkamp Micha de Winter Michele de Waard Michele Tribalat Milli Gorus Mimount Bousakla Mirjam de Rijk Mohammed Arkoun Mohammed Benzakour Mohammed Cheppih Mohammed Iqbal Mohammed Reza Pahlavi Mohammed Sini Naema Tahir Naima Elmaslouhi Nazih Ayubi Nazmiye Oral NCB Neal Ascherson Nebahat Albayrak Nico de Haas Nico van Nimwegen Nieuw Israelitisch Weekblad Norman Podhoretz Nout Wellink NRC Handelsblad Olivier Roy Orhan Pamuk Oscar Garschagen Oscar Hammerstein Osdorp Posse Paul Cliteur Paul de Beer Paul Gerbrands Pauline Meurs Paul Meurs Paul Rosenmuller Paul Scheffer Paul Schnabel Peter Langendam Peter Smit Pierre Bourdieu Pierre Heijnen Pieter Lakeman Piet Hein Donner Pim Fortuyn PPR President Eisenhower Prinses Maxima Profeet Mohammed Professor Peters PvdA Qotb Ramsey Nasr Renate Rubinstein Rene Cuperus RIAGG Ria van Gils Rita Verdonk Rob Oudkerk Roger van Boxtel Ronald van Raak Rudy Kousbroek Ruud Koopmans Ruud Lubbers Saddam Hussein Said Benayad Salman Rushdie Samuel Huntington Schelto Patijn SCP Senay Ozdemir Shervin Nekuee Shukri Silvio Berlusconi Simone van der Burg SISWO Sjaak van der Tak Sjeik Ahmed Yassin Sjoerd de Jong Soedish Verhoeven SP Spinoza Stef Blok Susan Moller Okin S.W. Couwenberg Sylvain Ephimenco Tara Singh Varma Theodor Adorno Theo van Gogh Theo Veenkamp Thijs WoltgensThorbecke Timothy Garton Ash Tiny Cox Tony Blair Tzvetan Todorov Van Montfrans Vlaams Blok VN Committee for elimination of all forms of racism Volkert van der Graaf VVD VVN Wasif Shadid Werner Schiffauer Wiardi Beckman Stichting Willem Vermeend Wim de Bie Wim Kok Wim Willems Wouter bos WRR WRR 1989 WRR 2001 WRR 2003 Yucel Yesilgoz Zekeriya Gumus Zeki Arslan Dutch Debate on Integration (2002-04),
  • 8. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Community Detection • Detect ‘natural’ communities in networks • Basic idea: ‘relatively’ many links inside communities
  • 9. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
  • 10. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free First principle approach • Reward “good” links, penalize “bad” links. • Assume contribution for links within and between are equal. • Simplifies to only internal links. Present Missing Within + − Between − + Link present or missing? Link within or between community? Reward (+) or Penalize (−)
  • 11. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free First principle approach • Reward “good” links, penalize “bad” links. • Assume contribution for links within and between are equal. • Simplifies to only internal links. Aij = 1 Aij = 0 δij = 1 aij −bij δij = 0 −cij dij Link present or missing? Link within or between community? Reward (+) or Penalize (−)
  • 12. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free First principle approach • Reward “good” links, penalize “bad” links. • Assume contribution for links within and between are equal. • Simplifies to only internal links. Aij = 1 Aij = 0 δij = 1 aij −bij δij = 0 −aij bij Link present or missing? Link within or between community? Reward (+) or Penalize (−)
  • 13. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free First principle approach • Reward “good” links, penalize “bad” links. • Assume contribution for links within and between are equal. • Simplifies to only internal links. Objective function H = − ij (aij Aij −bij (1 − Aij )) δij
  • 14. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Null-model Objective function H = − ij (aij Aij −bij (1 − Aij )) δij • Introduce weights aij = 1 − γRBpij and bij = γRBpij . • Null-model pij , constrained by ij pij = 2m. • Parameter γ known as resolution parameter: Higher γ ⇒ smaller communities. Lower γ ⇒ larger communities. • Leads to HRB = − ij (Aij − γpij ) δij .
  • 15. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Configuration Null-model Reichard-Bornholdt objective function HRB = − ij (Aij − γpij ) δij • Rewire links, but keep degrees unchanged • Probability link between i and j is then pij = ki kj 2m . i jki kj
  • 16. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Configuration Null-model Reichard-Bornholdt objective function HRB = − ij (Aij − γpij ) δij • Rewire links, but keep degrees unchanged • Probability link between i and j is then pij = ki kj 2m . i jki kj γ = 1 leads to modularity Q = 1 2m ij Aij − ki kj 2m δij ∼ −H.
  • 17. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Basic properties Modularity Q = 1 2m ij Aij − ki kj 2m δij • Normalized: between −1 (bad partition) and 1 (good partition). • Trivial partitions have modularity Q = 0. • No community consists of a single node. • Each community is internally connected. • Modularity maximization is NP-complete.
  • 18. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
  • 19. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Examples: grid Modularity tends to 1.
  • 20. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Examples: tree Modularity tends to 1.
  • 21. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Examples: random graph Random partition has Q ∼ 0, but best partition has Q ∼ √ k k > 0.
  • 22. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Problem 1: Incomparability Modularity can be high even “without communities”
  • 23. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Compare modularity scores (Dis)agreement on whether smoking is cancerous.
  • 24. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Scientific specialization
  • 25. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Scientific specialization
  • 26. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Scientific specialization
  • 27. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
  • 28. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: ring of cliques Modularity might merge cliques
  • 29. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: ring of rings Modularity might split rings
  • 30. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: Cliques of different size No longer fix γ = 1: tune γ so that it finds “correct” partition. No γ yields “correct” partition.
  • 31. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: broad community sizes Competing constraints: • Lower γ ⇒ merge smaller cliques. • Higher γ ⇒ split large cliques. No γ yields “correct” partition.
  • 32. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: broad community sizes Competing constraints: • Lower γ ⇒ merge smaller cliques. • Higher γ ⇒ split large cliques. No γ yields “correct” partition.
  • 33. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Example: broad community sizes Competing constraints: • Lower γ ⇒ merge smaller cliques. • Higher γ ⇒ split large cliques. No γ yields “correct” partition.
  • 34. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Problem 2: Resolution-limit Size of communities might depend on size of network
  • 35. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Outline 1 Introduction 2 Modularity 3 Problem 1: Incomparability 4 Problem 2: Resolution-limit 5 Resolution-limit-free
  • 36. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Resolution limit revisited Resolution-limit Resolution-limit-free • Problem is not merging per s´e. • Rather, cliques separate in subgraph, but merge in large graph (or vice versa).
  • 37. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Resolution limit revisited Resolution-limit Resolution-limit-free Definition (Resolution-limit-free) Objective function H is called resolution-limit-free if, whenever partition C optimal for G, then subpartition D ⊂ C also optimal for subgraph H(D) ⊂ G induced by D.
  • 38. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Resolution-limit methods General community detection H = − ij (aij Aij −bij (1 − Aij )) δij Not resolution-limit-free RB model Set aij = 1 − γpij , bij = γRBpij . Modularity Set pij = ki kj 2m and γ = 1 What weights aij and bij to choose so that model is resolution-limit-free?
  • 39. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Resolution-limit-free methods General community detection H = − ij (aij Aij −bij (1 − Aij )) δij Resolution-limit-free RN model Set aij = 1, bij = γRN. CPM Set aij = 1 − γ and bij = γ. Leads to H = − ij (Aij − γ)δij . Are there any other weights aij and bij ?
  • 40. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Local weights General community detection H = − ij (aij Aij −bij (1 − Aij )) δij i j Definition (Local weights) Weights local when they only depend on subgraph induced by i and j.
  • 41. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Local weights General community detection H = − ij (aij Aij −bij (1 − Aij )) δij i j Theorem (Local weights ⇒ resolution-limit-free) Method is resolution-limit-free if weights are local.
  • 42. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Local weights General community detection H = − ij (aij Aij −bij (1 − Aij )) δij i j Are local weights necessary?
  • 43. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free “Almost” resolution-free ⇒ local weights α1 α2 α3 β1 β2 β3 β4 Not resolution free, whenever merged in large graph, but separate in subgraph, or the other way around. So Hm < Hs and Hm > Hs, or Hm > Hs and Hm < Hs
  • 44. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free “Almost” resolution-free ⇒ local weights α1 α2 α3 β1 β2 β3 β4 Working out, yields α3(β4(nc − 1) + 2β3) < α3(β4(nc − 1) + 2β3) or, α3(β4(nc − 1) + 2β3) > α3(β4(nc − 1) + 2β3). Always satisfied for non-local weights, except when (nc − 1) = 2 β3α3 − β3α3 β4α3 − β4α3 .
  • 45. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
  • 46. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
  • 47. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
  • 48. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
  • 49. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Louvain like algorithm 1 1 1 1 1 1 1 1 1 1 57 5 6 2 1 Init node sizes ni = 1 2 Loop over all nodes (randomly), calculate improvement ∆H(σi = c) = (ei↔c − 2γni j nj δ(σj , c)), 3 Create community graph, repeat procedure
  • 50. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Performance (directed networks) µ0 0.2 0.4 0.6 0.8 1.0 NMI 0.25 0.5 0.75 1 CPM γ=γ∗ ER γ=p RB Conf γRB=γ∗ RB Mod. γRB=1 Inf. n = 103 n = 104
  • 51. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Problems of CPM No problems: • Doesn’t merge cliques. • Doesn’t split rings. • Correctly detects cliques of different size. Remaining problem: • Communities of different density.
  • 52. Introduction Modularity Problem 1: Incomparability Problem 2: Resolution-limit Resolution-limit-free Conclusions • Modularity high even when no communities. • Modularity merge/splits communities unexpectedly (resolution-limit). • Methods using local weights are resolution-limit-free. • Resolution-limit-free method performs superbly. Open question: • what when communities have different densities? Thank you for your attention. Questions?