The document proposes a method for visual localization of autonomous robots using color histogram-based clustering feature (CF) trees. The method extends previous work on an autonomous mobile robot that models color information using online clustering. It solves the localization problem by comparing the robot's CF tree built incrementally from observed data to reference CF trees constructed from model data in different areas. The comparison uses an all common subsequence measure between paths in the trees. Experiments show the method can localize the robot in different environments and lighting conditions in under 550 milliseconds using a desktop PC. Future work will investigate more robust features and improving color-based features' stability to illumination changes.

1. Visual Impression Localization of
Autonomous Robots
Somar Boubou, A.H. Abdul Hafez, Einoshin Suzuki
1. Dept. of Informatics, Kyushu University, Japan.
2. Control Systems Laboratory, Toyota Technological Institute, Japan.
3. Dept. of Computer Engineering, Hasan Kalyoncu University, Turkey.
1
1,2 13

2. Topological visual localization:
• Appearance-based methods:
2
• Landmark-based methods:
[ Pronobis 06]
https://www.dyson360eye.com

3. Previous Localization methods are precise = every node
in the topological map represents a (relatively) precise
position of the robot. [Abdul-Hafez13]
Precision: around 1m in outdoor applications, order of
mm in indoor applications when geometric features are
available. [Badino12][BK Kim15]
3
We achieved a
rough but fast
localization with BIRCH.
Background and Objective

4. Base Work: Autonomous Mobile
Robot that Models HSV Color Info. of
the Environment [Suzuki 2012]
Navigating indoor, the robot uses online clustering
BIRCH [Zhang 97] and detects peculiar colors
4
X4

5. Proposed extension to our
localization problem
• Robot in [Suzuki12] signals an observation which is
sufficiently far from similar past observations
• Our robot inherits most of [Suzuki 12] but solves a
localization problem by comparing a pair of CF trees
based on All Common Sequence [Wang 97]
5
Observed
data
CF tree
on RAM
Incremental construction of
the model
Leaf: compressed
similar observations
Outlier (very different from
the corresponding leaf)

6. Localization problem
Ref1
Nav
Ref4
Ref3
Ref2
Nav CF-tree
Ref CF trees
on ROM
?
Robot localize itself by comparing its
tree with several reference trees.
Each of which is representing one
area of interest.
6

7. Localization problem
Ref1
Nav
Ref4
Ref3
Ref2
Nav CF-tree
Ref CF trees
on ROM
Robot localize itself by comparing its
tree with several reference trees.
Each of which is representing one
area of interest.
6

8. Localization problem
Ref1
Nav
Ref4
Ref3
Ref2
Nav CF-tree
Ref CF trees
on ROM
Robot localize itself by comparing its
tree with several reference trees.
Each of which is representing one
area of interest.
6

9. BIRCH [Zhang 97]
BIRCH, Balanced Iterative Reducing and Clustering using
Hierarchies:
• Groups similar examples by building a data index
structure called a CF tree (i.e., Clustering Feature tree).
• An efficient and scalable clustering method for a huge
data set. [Zhang 97]
Applications:
• Peculiar data discovery [Suzuki 12] and intrusion
detection [Horng 11]
7

10. The Clustering Feature 𝐶𝐹 of a
cluster 𝕏 is a triple, denoted as:
CF tree [Zhang 97] 8
N d-dimensional data points or
feature vectors x1, x2, … , x 𝑁
Cluster 𝕏
𝐿𝑆 = x 𝑖
𝑁
𝑖=1 𝑆𝑆 = x 𝑖
2𝑁
𝑖=1
𝐶𝐹 = 𝑁, 𝐿𝑆, 𝑆𝑆

11. 𝑑 𝐶𝐹𝑥, 𝐶𝐹𝑖 < τ
CF vector
[Zhang97]
9
𝐶𝐹𝑖 ⊕ 𝐶𝐹𝑥 = 𝑁𝑖 + 𝑁𝑥, 𝐿𝑆𝑖 + 𝐿𝑆 𝑥, 𝑆𝑆𝑖 + 𝑆𝑆 𝑥
𝐶𝐹𝑥 = 𝑁𝑥, 𝐿𝑆 𝑥, 𝑆𝑆 𝑥 𝐶𝐹𝑖 = 𝑁𝑖, 𝐿𝑆𝑖, 𝑆𝑆𝑖
insert
Yes
No
Try again in new location

12. CF Vector for HSV color histogram
[Suzuki 12]
𝐶𝐹 = 𝑁, 𝐿𝑆, 𝑆𝑆
𝐶𝐹 = (ℎ, 𝝎, 𝑛𝑢𝑚, 𝑘𝑒𝑦)
10
𝑘𝑒𝑦 = [𝐵; 𝐺; 𝑊; 𝑟0:3; 𝑜0:3; 𝑦0:3; 𝑔0:3; 𝑐0:3; 𝑏0:3; 𝑝0:3]
Our extension: introduction
of weights
[Lei 99]

13. Robot
Navigation
Navigation tree (𝒯)
Paths of tree
1 P
Reference tree (𝒮)
Paths of tree
1 Q
Navigation tree (𝒯)
Comparison
of the paths
Arrangement of the
comparison results
δ(1,1)= …
δ(1,2)= …
.
.
δ(P,Q)= …
δ(1)> δ(2)>…> …> δ(P.Q)
S(𝒮,𝒯) = 𝜸 δ(𝒙)
𝑷.𝑸
𝟏
Similarity(𝒮,𝒯) =
𝑺(𝓢,𝓣)
𝑺(𝓣,𝓣)
𝒮= 𝑆1, 𝑆2, ⋯ , 𝑆 𝑃
𝒯= 𝑇1, 𝑇2, ⋯ , 𝑇𝑄
𝛿 𝑎𝑐𝑠(𝑝, 𝑞) =
𝑎𝑐𝑠 𝑆 𝑝, 𝑇𝑞
2 𝑀+𝑁
ω 𝑎𝑐𝑠
𝜸 = 𝑸 𝟐/𝑷𝑸 𝑖𝑠 𝑎 𝑠𝑐𝑎𝑙𝑖𝑛𝑔 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒.
Flow Chart for CF-trees comparison

14. Node weighting
𝛿 𝑎𝑐𝑠 =
𝑎𝑐𝑠 𝑠, 𝑡
𝑎𝑐𝑠 𝑠, 𝑠 𝑎𝑐𝑠 𝑡, 𝑡
𝝎 𝒂𝒄𝒔 → 𝛿 𝑎𝑐𝑠 =
𝑎𝑐𝑠 𝑠, 𝑡
2 𝑀+𝑁
𝝎 𝒂𝒄𝒔
𝑠 = 𝑆 𝑝, 𝑡 = 𝑇𝑞
ω(𝑖) =
αω 𝑛 + βω 𝑝𝑜𝑠
α + β
Let us consider two paths:
12
Weights are used:
- to define compression type.
- to eliminate noise.
𝝎 𝒂𝒄𝒔 = 1 −
max ω 𝑠, ω 𝑡 − min(ω 𝑠, ω 𝑡)
max(ω 𝑠, ω 𝑡)
ω 𝑠 = ψω(𝑖)𝑀
𝑖=1
[Wang 97]:

15. Node weighting
𝛿 𝑎𝑐𝑠 =
𝑎𝑐𝑠 𝑠, 𝑡
𝑎𝑐𝑠 𝑠, 𝑠 𝑎𝑐𝑠 𝑡, 𝑡
𝝎 𝒂𝒄𝒔 → 𝛿 𝑎𝑐𝑠 =
𝑎𝑐𝑠 𝑠, 𝑡
2 𝑀+𝑁
𝝎 𝒂𝒄𝒔
𝑠 = 𝑆 𝑝, 𝑡 = 𝑇𝑞
ω(𝑖) =
αω 𝑛 + βω 𝑝𝑜𝑠
α + β
Let us consider two paths:
12
Weights are used:
- to define compression type.
- to eliminate noise.
𝝎 𝒂𝒄𝒔 = 1 −
max ω 𝑠, ω 𝑡 − min(ω 𝑠, ω 𝑡)
max(ω 𝑠, ω 𝑡)
ω 𝑠 = ψω(𝑖)𝑀
𝑖=1
s={a,b,c}
t={a,b}
acs(s,t)={∅, 𝑎, 𝑏, 𝑎𝑏}= 4
[Wang 97]:

16. Node weighting
𝛿 𝑎𝑐𝑠 =
𝑎𝑐𝑠 𝑠, 𝑡
𝑎𝑐𝑠 𝑠, 𝑠 𝑎𝑐𝑠 𝑡, 𝑡
𝝎 𝒂𝒄𝒔 → 𝛿 𝑎𝑐𝑠 =
𝑎𝑐𝑠 𝑠, 𝑡
2 𝑀+𝑁
𝝎 𝒂𝒄𝒔
𝑠 = 𝑆 𝑝, 𝑡 = 𝑇𝑞
ω(𝑖) =
αω 𝑛 + βω 𝑝𝑜𝑠
α + β
Let us consider two paths:
12
Weights are used:
- to define compression type.
- to eliminate noise.
𝝎 𝒂𝒄𝒔 = 1 −
max ω 𝑠, ω 𝑡 − min(ω 𝑠, ω 𝑡)
max(ω 𝑠, ω 𝑡)
ω 𝑠 = ψω(𝑖)𝑀
𝑖=1
[Wang 97]:
ω 𝑛 𝑖 =
𝑛𝑖
𝑛 𝑟𝑜𝑜𝑡
ω 𝑝𝑜𝑠 =
𝑣
3

17. Tree types of comparison
(favor of root)
𝑘𝑒𝑦 = [𝐵; 𝐺; 𝑊; 𝑟0:3; 𝑜0:3; 𝑦0:3; 𝑔0:3; 𝑐0:3; 𝑏0:3; 𝑝0:3]
49_𝑘𝑒𝑦 𝑟𝑜𝑜𝑡 = [𝑊: 57%; 𝐵: 43%; ]
21_𝑘𝑒𝑦 𝑐ℎ𝑖2 = [𝐵: 100%; ]
28_𝑘𝑒𝑦𝑐ℎ𝑖1 = [𝑊: 100%; ]
49_𝑘𝑒𝑦 𝑟𝑜𝑜𝑡 = [𝑊: 57%; 𝑟0: 43%; ]
28_𝑘𝑒𝑦𝑐ℎ𝑖1 = [𝑊: 100%; ]
21_𝑘𝑒𝑦 𝑐ℎ𝑖2 = [𝑟0: 100%; ]
13

18. Tree types of comparison
(favor of leaves)
𝑘𝑒𝑦 = [𝐵; 𝐺; 𝑊; 𝑟0:3; 𝑜0:3; 𝑦0:3; 𝑔0:3; 𝑐0:3; 𝑏0:3; 𝑝0:3]
49_𝑘𝑒𝑦 𝑟𝑜𝑜𝑡 = [𝑊: 90%; ]
[𝑟0; 𝑜0; 𝑦0; 𝑔0; 𝑐0; 𝑏0; 𝑝0 = 2% < 5%]
1_𝑘𝑒𝑦 𝑐ℎ𝑖2 = [𝐵: 100%; ]
44_𝑘𝑒𝑦𝑐ℎ𝑖1 = [𝑊: 100%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖3 = [𝐺: 100%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖6 = [𝑏0: 100%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖4 = [𝑦0: 100%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖5 = [𝑐0: 100%; ]
49_𝑘𝑒𝑦 𝑟𝑜𝑜𝑡 = [𝑊: 90%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖2 = [𝐵: 100%; ]
44_𝑘𝑒𝑦𝑐ℎ𝑖1 = [𝑊: 100%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖3 = [𝐺: 100%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖6 = [𝑏0: 100%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖4 = [𝑦0: 100%; ]
1_𝑘𝑒𝑦 𝑐ℎ𝑖5 = [𝑐0: 100%; ]
In 𝑘𝑒𝑦 𝑟𝑜𝑜𝑡 :
14

19. Experiments (1)
Favor of the
root
Favor of the
Leaves
Neutral
- Six areas.
- One reference tree for
each area.
- Five navigation trials in
each area.
- Three types of comparison were
introduced:

20. Results (1)
Favor of the root Favor of the LeavesNeutral

21. Experiments (2): KTH-IDOL2 Dataset
[ Pronobis06]
- 5 rooms and three illumination conditions
which are, cloudy, night, and sunny.
17
Four navigation trials under each condition:
- Three trials were used to create reference CF trees.
-The forth trials were used to create navigation trees.

22. KTH-IDOL2
Results (2)
0%
20%
40%
60%
80%
Cloudy Night Sunny
Training /Cloudy/
CAMML
NBM
Filter
0%
20%
40%
60%
80%
Cloudy Night Sunny
Training /Night/
0%
20%
40%
60%
80%
Cloudy Night Sunny
Training /Sunny/
[Rubio 14]
- /CAMML/ Bayesian Network
- Naive Bayes Method

23. 19
- PC with 32-bit Ubuntu 12.04 system.
- Equipped with Intel Core i7 CPU 920.
- Clock speed: 2.67GHz.
- RAM: 11.8GB.
Computation time
/Our Platform/

24. Computation time
20
𝑡 𝑎 =29ms
per frame
Paths of tree
1 P
Reference tree (𝒮)
Paths of tree
1 Q
Navigation tree (𝒯)
Compare
𝑇𝑐 = 𝑡 𝑐 𝑃𝑄
𝑡 𝑐 =0.031 ms, P=Q=60
𝑇𝑐 = 111.56 𝑚𝑠
15 fps 𝑄 ≈ 60
𝑇𝑎 = 𝑡 𝑎∗ 60 = 435 𝑚𝑠
𝑇𝑡𝑜𝑡 = 𝑡 𝑎 + 𝑡 𝑐 = 546.56 𝑚𝑠
320×240

25. Contributions
• We are planning to investigate more robust features
(e.g., SIFT, SURF, WI-SURF, HOG) to the changes in
the environment due to illumination etc.
21
• Extend the discovery robot [Suzuki 12] for our
localization problem.
• Color-based feature were not stable under different
illumination conditions.
• Introduce a new measure for CF-tree similarity based
on ACS.
Future work

26. Thank you for listening…