This document summarizes a research paper that proposes segmenting images into objects by representing the image as a graph and using the Content Map Equation (CME) algorithm. The CME treats each pixel as a node in the graph and partitions the nodes to segment the image. The algorithm is tested on simple black and white images with added noise. Results show the CME achieves higher accuracy than K-means clustering in segmenting objects, though it sometimes partitions single objects across multiple segments. Future work involves merging CME segments to improve object segmentation.

1. Segmenting Images through Representation as a Content-Rich Network
Author: Yanira Corvera, Advisor: Dr. Laura Smith
Department of Mathematics, California State University Fullerton
Abstract
Image segmentation is the process consisting of partitioning digital images into multiple
sections with the goal of detecting objects within the image. In this work, we represent an
image as a graph, where each pixel is a node in the graph and an edge is present between
two nodes if they are close spatially within the image. In particular, we treat the image
as a content-rich graph, where each node is associated with a vector of different features
of the image, such as RGB and HSV color representations and textures. We consider
graph cut and clustering methods to partition the nodes, resulting in a segmented image.
We use the information theoretic approach, the Content Map Equation, which minimizes
the description length of the network by compressing the graph into distinct modules
of nodes, giving the partitioning of the image. We compare this method to existing
graph-cut methods, such as normalized cut, on a variety of images. Implementation
of the Content Map Equation on a simple black and white image provides promising
results which encourage further investigation on more complex images.
Introduction
• We want to partition a given image.
• An image is represented as a network with pixels as the nodes.
• Each node has an associated feature vector, e.g. RGB.
• A dictionary vector is created from the feature values.
• The image is segmented by using graph-cut methods
Methods
• Weights of edges between nodes are calculated by:
wij = e
−||F(i)−F(j)||
σF
feature
distance
∗ e
−||X(i)−X(j)||
σX
spatial
distance
,
if ||X(i) − X(j)|| < r, and wij = 0 otherwise [1].
• For an RGB feature vector, we define 8 words per color band
and create dictionary vectors of length 24, e.g.
Fv = [0, 65, 198] → {da
j} = [10000000|00001000|00000001].
• The 8 words are determined by the CDF of the color band,
giving each word an equal probability of 1/8 of occuring.
Figure 1: The CDF of an image’s red color band that is utilized
to define words for the dictionary vector.
Segmented Image Results of Toy Image with Noise
(a) Noisy Image with
= 0
(b) Content Map Equation Results (c) K-Means
Clustering Results
(d) Noisy Image with
= 0.00006
(e) Content Map Equation Results (f) K-Means
Clustering Results
(g) Noisy Image with
= 0.00008
(h) Content Map Equation Results (i) K-Means
Clustering Results
(j) Noisy Image with
= 0.0001
(k) Content Map Equation Results (l) K-Means
Clustering Results
Figure 2: Partitions of Noisy Toy Images with the Implementation of the Content Map Equation (CME) and K-Means Clustering Method
on K eigenvectors. These are the resulting segmented noisy toy images where we defined 8 words for the CME and used constraints F = 120 and R = 24
with σF = 72 and σX = 98. The partitions resulting from the K-Means Clustering method on K eigenvectors required the constraints F = 176, R = 10, and
K = 3 with σF = 5 and σX = 4.
Graph Cut Methods
We use the Content Map Equation (CME) [2] to partition
the graph. This is done by using a bottom-up approach [3] to
find a local minimum of
LC(M) = ( q H(Q))
contribution of
movement
between
modules
+


m
i=1
pi
H(Pi
)


contribution of
movement
within
modules
+


m
i=1
pi
H(Xi
)


contribution
of node
dictionary
vectors
.
• H(Q) is the entropy of the movement between modules.
• H(Pi
) is the entropy of movement within a module.
• H(Xi
) is the entropy of movement between dictionary words.
• These terms are weighted by the frequencies of
• exiting a module, q
• exiting or remaining in the module, m
i=1 pi
• and remaining in the module, m
i=1 pi .
• The goal is to decrease the description length of a network by
clustering nodes based on distance and dictionary similarity.
For comparison, we use a K-Means Clustering method:
• We find the k eigenvectors associated with the k smallest
eigenvalues of the symmetric normalized Laplacian matrix [1],
Ls = D−1
2(D − W)D−1
2,
where W is the weighted adjacency matrix and D is a
diagonal matrix with Dii = j Wij, the sum of the edge
weights adjacent to node i.
• The eigenvectors then create points in Rk
, which are
partitioned into k clusters by applying the k−means
clustering algorithm.
• The resulting partitioning gives k clusters of nodes. k is
predetermined.
Metrics for Comparison
• We use Normalized Mutual Information (NMI) and Purity
measures to compare the partitioning results to the ground
truth communities. Values of 1.0 are optimal.
• NMI is calculated by:
NMI =
I(U, V )
H(U)H(V )
,
where I(U, V ) is the mutual information between sets U and
V , H(U) is the entropy of a set U, and H(V ) is the entropy
of a set V .
• Purity of one cluster is calculated by:
P(Cj) =
1
|Cj|
max
k=1,...,c
|Cj,k|,
where Cj,k consists of the cluster Cj of an image that
originally belongs to the class set k.
Results
• Noise was added to a toy image. The epsilon values indicate
the amount of noise.
• The CME produces higher NMI values than the K-Means
Clustering method for = 0, 0.00003 − 0.00005, & 0.00007.
• CME has higher purity for = 0.00002 − 0.00005, & 0.00007.
• Clusters resulting from the CME model have a larger number
of nodes in each cluster belonging to a particular class than
the K-Means Clustering method.
• One advantage of CME model is the number of communities
does not have to be predetermined, but this may be a dis-
advantageous when partitioned into too many clusters.
Evaluation of Partitions with Groundtruth
(a) NMI Measure
(b) Purity Measure
Figure 3: NMI and Purity Measures. This gives a comparison of the
Content Map Equation partitions with those of the K- Means Clustering
method. The purity results are impressive for CME partitions since the
model distinguished the distinct objects of the image. However, its NMI is
low since the objects, for example the rectangle, are being clustered into
several distinct communities as opposed to the optimal full object.
Conclusion
• High purity values for the CME model indicate the method is
able to distinguish the different objects.
• With a low NMI, the CME model puts the different objects in
distinct clusters consisting of several smaller communities.
• Lower purity values for the K-Means Clustering method
suggest the method is not able to fully distinguish the objects.
Discussion/Future work
• To address the multiple communities for a single object, we
will extend the method by treating communities resulting
from the CME as new nodes that we want to merge.
• We would like to experiment with different edge weights and
dictionary definitions.
• We would like to examine the approach on more complex
images.
References
1 Shi, J., & Malik, J. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Vol. 22. No. 8. p. 888–905. 2000.
2 Smith, L., Zhu, L., Lerman, K., & Percus, A. (2014). Partitioning networks with node
attributes by compressing information flow. Unpublished manuscript.
3 Rosvall, M., & Bergstrom, C. T.. Proceedings of the National Academy of Sciences. Vol.
105. No. 4. p. 1118–1123. Jan. 2008.
4 Danon, L., Diaz-Guileram, A., Duch, J. & Arenas, A. Journal of Statistical Mechanics:
Theory and Experiment. 2005.
5 Strehl, A, & Ghosh, J.. Journal of Machine Learning Research. Vol. 3. p. 583–617. 2002.
6 Huang, A.Similarity Measures for Text Document Clustering. Proceedings of the New
Zealand Computer Science Research Student Conference. 2008.
7 Chen,Y., Wang, J. Z., & Krovetz, R.. IEEE Transactions on Image Processing. Vol.14.
No. 8. p. 1187–1201. 2005.
8 Chen, M. Normalized Mutual Information Code. March 2012. Online. May 2014.
http://www.mathworks.com/matlabcentral/fileexchange/35625-information-theory-
toolbox/content/nmi.m.
9 Smith, L., Zhu, L., Lerman, K., & Percus, A.The Content Map Equation Code. June
2013.