G-MSM: Unsupervised Multi-Shape Matching with Graph-based Affinity Priors

1. G-MSM: Unsupervised Multi-
Shape Matching with Graph-
based Affinity Priors
Tien-Bach-Thanh Do
Network Science Lab
Dept. of Artificial Intelligence
The Catholic University of Korea
E-mail: osfa19730@catholic.ac.kr
2024/05/20
Marvin Eisenberger et al.
CVPR 2023

2. 2
Introduction
● Shape matching of non-rigid object categories is a central problem in 3D CV
● The majority of existing DL methods for shape matching treat a given set of meshes as an unstructured
collection of poses
● Random pairs of shapes are sampled for NN and pairwise matching loss is minimized
● Fail to recognize commonalities and context-dependent patterns
● Not all samples of a shape collection are created equal, some pairs of poses are much closer than others
● Define an affinity graph, undirected shape graph G on the set of input shapes whose edge weights
(affinity scores) are informed by the outputs of pairwise matching module
● Define multi-matching architecture that propagates matches along shortest paths in the underlying shape
graph G, apply cycle-consistency for optimal paths in the G

3. 3
Introduction

4. 4
Related Works
● Axiomatic correspondence methods
● Learning-based methods
● Multi-shape matching

5. 5
Method
Problem formulation
● Consider a collection of 3D shapes S = {X(1),…, X(N)} from non-rigidly deformable shape categories
● Each shape is a discretized approximation of 2D Riemannian manifold
● X(i) = (V(i), T(i)) where V is set of nodes and T is triangular faces
● Goal is construct an algorithm computes dense correspondence mapping between any two surfaces from
the shape collection
● Demonstrate proposed multi-matching approach excels, including non-isometric pairs, poses with
topological noise from self-intersections, and inter-class matching

6. 6
Method
Network architecture

7. 7
Method
Network architecture
● Consist of 3 separate components, first 2 modules are standard components namely learable feature
backbone I and a pairwise matching layer II, finally the multi-matching architecture III
● Feature extractor defined as
given input shape X(i) = (V(i), T(i)), the mapping produces an l-dimensional feature embedding F per node V
(DiffusionNet)
● Pairwise matching is multi-scale matching scheme based on DeepShells
Given transport plan, the energy specifies the distance between the discrete measures associated with 2
arbitrary l-dimensional feature embeddings F and G
Take the minimum overall possible transport plans results in Kantorovich formulation of optimal transport
Phi match is defined as a deterministic, differentiable function that takes local feature encodings F as input
and predicts a set of correspondences II
These coordinates specify a registered version of the first input shape that closely aligns with the pose of
second input shape. l>0 is a training loss signal

8. 8
Method
Network architecture
● Graph-based multi-shape matching
○ Shape graph: construct G as a complete graph (undirected, fully connected), where missing edge
between X(i) and X(j) can be specified equivalently by setting edge weight
○ Define pairwise edge weights represent affinity scores between pairs of shapes
○ Propose heuristic for a given pair of shapes and define symmetric affinity score w as
small matching energy implies a high geometric similarity between the input poses
● Multi-matching
○ Obtain multi-shape matches

9. 9
Method
Network architecture
● Graph-based multi-shape matching
○ Pass along shortest paths in the graph, the approach thereby favors edges with close affinity (small
pairwise matching cost)
○ Promote cycle-consistency during training via the loss
This loss imposes a penalty on inconsistencies between the registration V produced by the pairwise matching
module, and the multi-shape correspondences
● Overall loss function

10. 10
Experimental Results
Dataset
● FAUST contains 10 humans in 10 different poses each
● SCAPE contains 71 diverse poses of the same individual
● SURREAL consists of synthetic SMPL meshes fitted to raw 3D motion capture data
● SHREC’19 Connectivity contains human shapes in different poses

11. 11
Experimental Results
Graph topologies
● Node is 3D shape
● Edge set = N(N-1)/2
○ Minimum spanning tree
○ Travelling salesman problem
○ Star graphs where all nodes are connected to one center node

12. 12
Experimental Results

13. 13
Experimental Results

14. 14
Experimental Results

15. 15
Experimental Results

16. 16
Experimental Results

17. 17
Experimental Results

18. 18
Conclusion
● G-MSM - a novel multi-matching approach for non-rigid shape correspondence
● Given collection of 3D meshes, define a shape graph G approximates the underlying shape data manifold
● Edge weights are extracted from putative pairwise correspondence signals in self-supervised manner
● Cycle-consistency of optimal paths, produces context-aware multi-matches informed by commonalities
and salient geometric features across all training poses