Correlation clustering with non convex optimization, to appear at AAAI 2019

1. 11/5/2018 Mikael Kågebäck, Chalmers CSE 1
Credit: Aldebaran
Correlation Clustering: A Tale of Two Cultures
Erik Thiel, Morteza Chehreghani, Devdatt Dubhashi
Chalmers University of Technology, Sweden

2. Are Colours Universal?

3. WCS Stimulus Palette

4. Colour Partitions
• Ask humans subjects in a language group to
label tiles with colour terms, then aggregate
all results into a partition for that language.
• Take the CIELAB colour coordinates to define a
similarity between colour tiles and forma
partition based on these similarities.

5. World Colour Survey

6. Regier T, Kemp C, Kay P. 2015.
Word meanings across languages
support efficient communication.
In
The Handbook of Language
Emergence, ed. B MacWhinney, W
O’Grady

7. Correlation Clustering
• Input: Graph 𝐺 = 𝑉, 𝐸 and positive or
negative weights 𝑤 𝑒 , 𝑒 ∈ 𝐸
• Output: A clustering of the vertices to
maximize the sum of the weights of edges
within each cluster.

9. Difference from usual Clustering
• Weights can be positive or negative!
• Contentious what’s ”good” quality clustering
• But in correlation clustering there is
unambiguous objective
• The number of clusters need not be specified,
will emerge from the optimizing the objective.

10. Web scale clustering

11. Approximation Algorithms
• Bansal, Blum Chawla (2004): PTAS on
complete graphs
• Charikar Guruswami, Wirth (2005): APX hard
on general graphs
• Charikar et al (2005), Swamy (2004): 0.76
approximation
• Guruswami-Giotis (2006): PTAS with fixed no
of clusters

12. Exact Formulation

13. SDP Relaxation + Rounding

14. However …
• No implementation, no code …
• Doesn’t work in practice …

15. A Tale of Two Cultures
• Deep elegant theory
• “Polynomial time”
• No implementation
• No experiments on data
sets
• Does not work in
practice or scale
• Beamer/LaTeX
• Sometimes theory
• Linear or sub-linear
• Well engineered
implementation
• Extensive testing on
data sets
• Must work in practice,
scale to “Big Data”
• Powerpoint
Algorithms Theory Machine Learning

16. Non-Convex Relaxation

17. Tightness of Relaxation
• The non-convex relaxation is tight: no gap
between continuous and discrete problem,
simple proof by randomized rounding.
• In contrast SDP relaxation is not tight.

20. Non-convex Convergence Theory
• For a differentiable (but not necessarily
convex) function, the FW algorithm converges
in 𝑂(1/ 𝑇) steps.
• If the function is multilinear, then it converges
in 𝑂(1/𝑇) steps.
• Note that our correlation clustering objective
is indeed multilinear!

21. Combinatorial Search

22. Synthetic Data: Generative Model
• Planted model with k clusters and noise p
• With probability (1-p), high positive weight on
edge within a cluster and high negative weight
on edge across clusters, with probability p,
arbitrary weight

26. Sampling Block FW

28. Variance Reduction Technique

29. Variance reduction

30. Newsgroup data set

31. Colour Partitions

32. Summary
• Non-convex relaxation solved with Frank
Wolfe yields an algorithm with guarantees
that beats all other methods handily in both
runtime and quality.
• Combine theory and rigour of algorithms
research with engineering good
implementations and extensive testing on
data.

33. Happy Birthday Anand!