The document presents 'CPlanet', a method for improving image geolocalization through combinatorial partitioning of maps, which enhances the prediction of geographical locations based on visual information. It combines results from multiple classifiers to create fine-grained geoclass sets, addressing the limitations of previous methods in handling complex class representations and data deficiencies. The proposed approach demonstrates improved performance over existing techniques across multiple evaluation benchmarks.

1. CPlaNet: Enhancing Image Geolocalization
by Combinatorial Partitioning of Maps
Paul Hongsuck Seo, Tobias Weyand, Jack Sim, Bohyung Han
hsseo@postech.ac.kr {weyand,jacksim}@google.com bhhan@snu.ac.kr

2. Problem Definition
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps
• Image Geolocalization
• Predicting geographic location of image based only on its visual
information.

3. Motivation (1)
• Geolocalization by Image Retrieval
• Conventional approach searching nearest neighbor in geo-tagged
image database.
• Geolocalization by Classification
• Recently proposed method classifying image over geoclasses
partitioning maps [1].
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps
[1] Weyand, T., Kostrikov, I., Philbin, J.: PlaNet-Photo Geolocation with Convolutional Neural Networks. In: ECCV. (2016)
Query photo

4. Motivation (2)
• Advantages
• Less complexity in time and space.
• Expressive power of multi-modal answer distribution.
• Limitations
• Artifacts from converting contiguous space into discrete class
representation.
• More classes ⇒ better approximation of contiguous space but more
severe data deficiency issue.
• No standard method to define geoclasses.
• Proposed Method: Combinatorial Partitioning
• Incorporating results from multiple classifiers facilitating
heterogeneous geoclass sets.
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

5. Proposed Method (1)
• Combinatorial Partitioning
• Aggregates results from multiple classifiers with heterogeneous,
coarse map partitions.
• Predicts results on fine-grained partitions.
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps
(𝑝1, 𝑞1)
(𝑝1, 𝑞2)
(𝑝1, 𝑞3)
(𝑝2, 𝑞3)
(𝑝2, 𝑞5)
(𝑝5, 𝑞5)
(𝑝5, 𝑞4)
(𝑝5, 𝑞3)
(𝑝3, 𝑞3)(𝑝3, 𝑞2)
(𝑝1, 𝑞1)
(𝑝4, 𝑞1)
(𝑝1, 𝑞1)
(𝑝3, 𝑞1)
𝑝5
𝑝2
𝑝3
𝑝1
𝑝4
Predictedscores
overgeoclassset𝒫
𝑞1
𝑞4
𝑞2
𝑞3
𝑞5
Predictedscores
overgeoclassset𝒬

6. Proposed Method (2)
• Benefits of Combinatorial Partitioning
• Fine-grained geoclasses with fewer parameters.
• More training data per class than naïve partitioning.
• More reasonable class sets facilitating heterogeneity.
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps
(𝑝1, 𝑞1)
(𝑝1, 𝑞2)
(𝑝1, 𝑞3)
(𝑝2, 𝑞3)
(𝑝2, 𝑞5)
(𝑝5, 𝑞5)
(𝑝5, 𝑞4)
(𝑝5, 𝑞3)
(𝑝3, 𝑞3)(𝑝3, 𝑞2)
(𝑝1, 𝑞1)
(𝑝4, 𝑞1)
(𝑝1, 𝑞1)
(𝑝3, 𝑞1)
𝑝5
𝑝2
𝑝3
𝑝1
𝑝4
Predictedscores
overgeoclassset𝒫
𝑞1
𝑞4
𝑞2
𝑞3
𝑞5
Predictedscores
overgeoclassset𝒬

7. Multiple Geoclass Sets Generation
• Initial Graph Construction:
Build initial graph where node represents regions, edge connects two
nodes of regions.
1. Collect non-empty S2 cells containing images as initial nodes.
2. Assign empty S2 cells to one of neighboring non-empty S2 cells
randomly to cover entire map.
3. Finally connect geographically adjacent initial nodes by edges.
• Geoclass Set Generation by Merging Nodes
• Node scores and edge weights:
• Hierarchically merge node with lowest score and its neighbor with
highest edge weight.
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

8. Multiple Geoclass Sets Generation
• Parameter Sampling for Generating Multiple Sets
• Two manually designed parameters
• Three randomly sampled parameters
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

9. Generated Class Sets
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

10. Network and Inference
• Network architecture
• Multiple classifiers with shared feature extractor.
• Learning by standard back-propagation with multi-task losses.
• Inference with combinatorial partitioning
• Aggregate scores from multiple classifiers for each S2 cell with
normalization.
• Return center location of S2 cells with highest score.
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps
Geoclass set 𝑁
…
Geoclass set 2
Geoclass set 1
Inception v3

11. Implementation and Evaluation
• Training
• Trained with large-scale dataset containing ~30.3M images.
• Asynchronous SGD with multiple workers to handle large dataset
size.
• Testing
• Speed up by indexing S2 cells of fine partitions and precomputing
centers of each partition.
• Evaluation Metric
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

12. Comparisons with Base Classifiers
• Results on Im2GPS3k
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

13. Comparisons with Other Methods
• Results on YFCC4k
• Results on Im2GPS
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

14. Analyses on Other Aspects
• Computational complexity
• 5 classifiers and combinatorial partitioning adds only 2% and 0.004% of theoretical time
complexity. Practical additional complexity is as below.
• Importance of visual features
• classifier balancing
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

15. Qualitative Results
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

16. Qualitative Results
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

17. Conclusion
• Proposed a novel classification-based approach for
image geolocalization.
• The model employs multiple classifiers and obtains the
final results on a fine-grained region given by
combinatorial partitioning.
• It improves performances over base classifiers and
other methods in multiple benchmarks.
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

18. CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps

19. CPlaNet: Enhancing Image Geolocalization
by Combinatorial Partitioning of Maps
Paul Hongsuck Seo, Tobias Weyand, Jack Sim, Bohyung Han
hsseo@postech.ac.kr {weyand,jacksim}@google.com bhhan@snu.ac.kr

20. Fine vs. Coarse Partitionings
CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps