What is a superpixel? This presentation describes Superpixel algorithms such as watershed, mean-shift, SLIC, BSLIC (SLIC superpixels based on boundary term) references: [1] Luc Vincent and Pierre Soille. Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6):583–598, 1991. [2] D. Comaniciu and P. Meer. Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5):603–619, May 2002. [3] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk, SLIC Superpixels Compared to State-of-the-art Superpixel Methods, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, num. 11, p. 2274 - 2282, May 2012. [4] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk, SLIC Superpixels, EPFL Technical Report no. 149300, June 2010. [5] Hai Wang, Xiongyou Peng, Xue Xiao, and Yan Liu, BSLIC: SLIC Superpixels Based on Boundary Term, Symmetry 2017, 9(3), Feb 2017.

1. Superpixel algorithms
Shima Foolad
shimafoolad@gmail.com
sh.foolad@Semnan.ac.ir
3 Dec. 2017
1

2. SUPERPIXEL ALGORITHMS
2

3. OUTLINE
Introduction1
2 Superpixel algorithms
1 Watershed
2
3 Visual Comparison of algorithms
Mean-shift
3 SLIC
3

4. WHAT IS A SUPERPIXEL?
A superpixel can be defined as a group of pixels which have similar
characteristics.
4

5. ADVANTAGES OF SUPERPIXEL
1. Good adherence to object boundaries
2. reduce the computational complexity from thousands of pixels to a few
hundred superpixels
5

6. APPLICATION OF SUPERPIXEL
segmentation
6

7. SUPERPIXEL ALGORITHMS
7
There are many algorithms to segment superpixels:
• Whatershed
• Mean-Shift
• Normalized-Cuts
• Graph-based
• QuichShift
• TurboPixels
• SLIC

8. WATERSHED ALGORITHM
8

9. WATERSHED ALGORITHM
9
• Any grey image can be considered as a topographic surface.
• flood this surface from its minima
• prevent the merging of the waters coming from different sources
Watershed algorithm was proposed by Vincent and Soille in 1991 Watershed lines

10. WATERSHED ALGORITHM
10
Watershed algorithm might be used on the gradient image instead of the original image.
Gradient imageOriginal image
Watershed of the original imageWatershed of the gradient image

11. WATERSHED ALGORITHM
11
in practice, this algorithm produces an over-segmentation due to noise or local irregularities in
the gradient image.

12. MEAN-SHIFT ALGORITHM
12

13. MEAN-SHIFT ALGORITHM
13
1. Choose a search window size.
2. Choose the initial location of the search window.
3. Compute the mean location (centroid of the data) in the search window.
4. Center the search window at the mean location computed in Step 3.
5. Repeat Steps 3 and 4 until convergence.
Mean-shift algorithm was proposed by Comaniciu and Meer in 2002

14. MEAN-SHIFT ALGORITHM
14

15. MEAN-SHIFT ALGORITHM
15

16. MEAN-SHIFT ALGORITHM
16

17. MEAN-SHIFT ALGORITHM
17

18. MEAN-SHIFT ALGORITHM
18
finding peaks in the high-dimensional data distribution

19. MEAN-SHIFT ALGORITHM
19
finding peaks in the high-dimensional data distribution in color image

20. MEAN-SHIFT ALGORITHM
20

21. MEAN-SHIFT ALGORITHM
22

22. SLIC ALGORITHM
23

23. SLIC ALGORITHM
Simple Linear Iterative Clustering (SLIC) algorithm
24
The Simple Linear Iterative Clustering (SLIC) algorithm was proposed by Radhakrishna Achanta in 2010

24. SLIC ALGORITHM
Sample segmentation output
25

25. SLIC WEB SITE
http://ivrl.epfl.ch/research/superpixels
26

26. SLIC
27
For color images, SLIC algorithm works in the LAB color space.

27. COVERT RGB TO LAB COLOR SPACE
28
𝐿 = 116. ℎ
𝑌
𝑌𝑤
− 16
𝐴 = 500 ℎ
𝑋
𝑋 𝑤
− ℎ
𝑌
𝑌𝑤
𝐵 = 200 ℎ
𝑌
𝑌𝑤
− ℎ
𝑍
𝑍 𝑤
ℎ 𝑞 = ቐ
3
𝑞 𝑞 > 0.008856
7.787𝑞 +
16
116
𝑞 ≤ 0.008856
𝑋
𝑌
𝑍
=
0.412453 0.357580 0.180423
0.212671 0.715160 0.072169
0.019334 0.119193 0.950227
𝑅
𝐺
𝐵
𝑋 𝑤=0.950450 𝑌𝑤=1.000000 𝑍 𝑤=0.088754

28. COVERT RGB TO LAB COLOR SPACE
B channel
Blue-yellow
A channel
Red-green
L channel
29
RGB

29. SLIC EXAMPLE
30
• Each pixel is assigned a 5-valued vector
(𝑋, 𝑌, 𝐿, 𝑎, 𝑏) for position and color
• Number of pixels: 𝑁 = 1600
• Image size: 40×40
• Number of superpixels: 𝐾 = 16

30. THE MAIN STEPS OF THE SLIC ALGORITHM
1. Initialize 𝐾 cluster centers 𝑪 𝒌 = 𝒍 𝒌, 𝒂 𝒌, 𝒃 𝒌, 𝒙 𝒌, 𝒚 𝒌
𝑻
on a regular grid spaced 𝑆
=
𝑁
𝐾
pixels apart.
31
• each superpixel has approximately
𝑁
𝐾
pixels

31. NUMBER OF SUPERPIXELS (K)
32

32. THE MAIN STEPS OF THE SLIC ALGORITHM
1. Initialize 𝐾 cluster centers 𝑪 𝒌 = 𝒍 𝒌, 𝒂 𝒌, 𝒃 𝒌, 𝒙 𝒌, 𝒚 𝒌
𝑻
on a regular grid spaced 𝑆
=
𝑁
𝐾
pixels apart.
33
• each superpixel has approximately
𝑁
𝐾
pixels
𝑁
𝐾
=
1600
16
= 100
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
𝑪1
𝑆
𝑆
𝑆 =
𝑁
𝐾
= 100 = 10
𝑪5
𝑪9
𝑪13
𝑪2
𝑪6
𝑪10
𝑪14
𝑪3
𝑪7
𝑪11
𝑪15
𝑪4
𝑪8
𝑪12
𝑪16

33. THE MAIN STEPS OF THE SLIC ALGORITHM
2. move these cluster centers to the positions with the lowest gradients in a 3 × 3
neighborhood;
This is done to avoid centering a superpixel on an edge, and to reduce to chance
of seeding superpixel with a noisy pixel.
34
𝐺 𝑥, 𝑦 = 𝐼 𝑥, 𝑦 − 1 − 𝐼(𝑥, 𝑦 + 1) 2 + 𝐼 𝑥 − 1, 𝑦 − 𝐼(𝑥 + 1, 𝑦) 2
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
𝑪1
𝑪5
𝑪9
𝑪13
𝑪2
𝑪6
𝑪10
𝑪14
𝑪3
𝑪7
𝑪11
𝑪15
𝑪4
𝑪8
𝑪12
𝑪16

34. THE MAIN STEPS OF THE SLIC ALGORITHM
2. move these cluster centers to the positions with the lowest gradients in a 3 × 3
neighborhood;
This is done to avoid centering a superpixel on an edge, and to reduce to chance
of seeding superpixel with a noisy pixel.
35
𝐺 𝑥, 𝑦 = 𝐼 𝑥, 𝑦 − 1 − 𝐼(𝑥, 𝑦 + 1) 2 + 𝐼 𝑥 − 1, 𝑦 − 𝐼(𝑥 + 1, 𝑦) 2
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
𝑪1
𝑪5
𝑪9
𝑪13
𝑪2
𝑪6
𝑪10
𝑪14
𝑪3
𝑪7
𝑪11
𝑪15
𝑪4
𝑪8
𝑪12
𝑪16
.

35. THE MAIN STEPS OF THE SLIC ALGORITHM
3. Assign pixels. Designate each pixel to a closest cluster center in a local search
space;
36
K-means algorithm SLIC algorithm

36. THE MAIN STEPS OF THE SLIC ALGORITHM
3. Assign pixels. Designate each pixel to a closest cluster center in a local search
space;
37
for each cluster center 𝑪 𝒌
Assign the best matching pixels from a 𝟐𝑺 × 𝟐𝑺 around the cluster center according to the distance measure.
end for
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
𝑪1
𝑪5
𝑪9
𝑪13
𝑪2
𝑪6
𝑪10
𝑪14
𝑪3
𝑪7
𝑪11
𝑪15
𝑪4
𝑪8
𝑪12
𝑪16

37. THE MAIN STEPS OF THE SLIC ALGORITHM
3. Assign pixels. Designate each pixel to a closest cluster center in a local search
space;
38
     
2 2 2
c j i j i j id l l a a b b     
   
2 2
s j i j id x x y y   
/S N K
2
2 2s
c
d
D d m
S
 
   
 
Distance measure
Color distance:
position distance:
𝐷′ =
𝑑 𝑐
𝑁𝑐
2
+
𝑑 𝑠
𝑁𝑠
2
𝐷′ =
𝑑 𝑐
𝑚
2
+
𝑑 𝑠
𝑆
2
m= Compactness of superpixel

38. COMPACTNESS OF SUPERPIXELS (m)
m = 1 m = 20 m = 40
39

39. THE MAIN STEPS OF THE SLIC ALGORITHM
3. Assign pixels. Designate each pixel to a closest cluster center in a local search
space by local KMC;
4. Update cluster centers. Set each cluster center as the mean of all pixels in the
corresponding cluster;
5. Repeat steps (3)–(4) until the clusters do not change or error (difference between
previous cluster and new cluster) is converge.
40
initialization iteration 1 iteration 2 iteration 3 iteration 4

40. THE MAIN STEPS OF THE SLIC ALGORITHM
6. Post-processing. The connected components algorithm is used to reassign
isolated regions to nearby superpixels if the size of the isolated regions is smaller
than a minimum size 𝑆 𝑚𝑖𝑛.
41
post-processing

41. SUPERVOXEL
42
2

42. SLICO
SLIC
SLICO
SLIC uses the same compactness parameter (chosen by user) for all superpixels in the image.
SLICO adaptively chooses the compactness parameter for each superpixel differently.
43

43. VISUAL COMPARISON OF ALGORITHMS
44
Watershed
1991
SLIC
2010
Mean-Shift
2002

44. VISUAL COMPARISON OF ALGORITHMS
45

45. ADVANTAGES OF SLIC
46
• low complexity
• generate superpixels with compact , regular size and shape
• SLIC is simple to use and understand

46. DISADVANTAGE OF SLIC
47
Turbopixels algorithmSLIC algorithmSuperPB algorithmSEEDS algorithm

47. BSLIC
48

48. BSLIC
49

49. BSLIC
50

50. DIFFERENCE BETWEEN SLIC AND BSLIC
51
• initializing cluster centers in hexagon rather than square
distribution
• choosing some specific edge pixels as cluster centers
• incorporating boundary term into the distance measurement during
k-means clustering

51. DISTRIBUTION OF CLUSTER CENTERS
52

52. DISTRIBUTION OF CLUSTER CENTERS
53

53. INITIALIZATION OF EDGE CENTERS
54
Edge-across superpixels. (a) Original image with slender rod; (b) Edge image with canny filter operator; (c) Partial
enlarged superpixels with only plane centers initialized in hexagon distribution; (d) Partial enlarged superpixels
with only plane centers initialized in square distribution.
hexagon distribution square distribution

54. PLANE CENTER AND EDGE CENTER
55
Plane center and edge center. Dots in solid black Ci0, Ci1, Ci2, Ci3, Ci4, Ci5 and Ci6 are “plane center”;
Square in solid red Ei1 is “edge center”.

55. hexagon distribution square distribution
Segmentation without
edge cluster center
Segmentation with
edge cluster center
56

56. DISTANCE MEASUREMENT
57

57. DISTANCE MEASUREMENT
58

58. Superpixel segmentation results
With edge cluster center
Segmentation without
edge cluster center
Segmentation with
edge cluster center
59

59. BSLIC ITERATING PROCESS
60
(a) Superpixels with only plane centers; (b) Superpixels with both plane and edge centers; (c)
First iterating result of BSLIC; (d) Final segmentation result of BSLIC.

60. BSLIC SUPERPIXELS SEGMENTATION RESULTS
61
m=5 m=15 m=30
k=100
k=500
k=1000

61. PART OF BSLIC EXPERIMENTAL RESULTS
62TurbopixelsSLICSuperPBSEEDSBSLIC

62. REFERENCES
[1] Luc Vincent and Pierre Soille. Watersheds in digital spaces: An efficient algorithm based on immersion
simulations. IEEE Transactions on Pattern Analalysis and Machine Intelligence, 13(6):583–598, 1991.
[2] D. Comaniciu and P. Meer. Mean shift: a robust approach toward feature space analysis. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 24(5):603–619, May 2002.
[3] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine
Süsstrunk, SLIC Superpixels Compared to State-of-the-art Superpixel Methods, IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 34, num. 11, p. 2274 - 2282, May 2012.
[4] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine
Süsstrunk, SLIC Superpixels, EPFL Technical Report no. 149300, June 2010.
[5] Hai Wang, Xiongyou Peng, Xue Xiao, and Yan Liu, BSLIC: SLIC Superpixels Based on Boundary Term,
Symmetry 2017, 9(3), Feb 2017.
63

63. Thanks
64