Internship presentation

1. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Surface reconstruction from point
clouds using optimal transportation
Guillaume Matheron
June-August 2015
Internship supervised by Pierre Alliez and David Cohen-Steiner
Guillaume Matheron Surface reconstruction from point clouds

2. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Input
Multiple and increasing variety of data sources
Guillaume Matheron Surface reconstruction from point clouds

3. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Drones
Guillaume Matheron Surface reconstruction from point clouds

4. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Satellites
Guillaume Matheron Surface reconstruction from point clouds

5. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Cars
Guillaume Matheron Surface reconstruction from point clouds

6. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Community data
Snavely [2009]
Guillaume Matheron Surface reconstruction from point clouds

7. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Acquisition
Stereography / Photogrametry
Depth cameras
Challenge : Noise and outliers
Guillaume Matheron Surface reconstruction from point clouds

8. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Surface reconstruction
Set of points → surface representation
Suited for simulation, storage, visualization...
Guillaume Matheron Surface reconstruction from point clouds

9. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Problem statement
Discrete set of points → 2-manifold surface
Ill-stated problem
Idea : Interpret both as mass distributions.
Error metric for mass distributions : Wasserstein distance
Guillaume Matheron Surface reconstruction from point clouds

10. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Optimal transportation
Wasserstein distance :
Wk(λ, µ)k
= inf d(x − y)k
dπ(x, y) | π ∈ Π(λ, µ)
Guillaume Matheron Surface reconstruction from point clouds

11. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Transportation as error metric
Digne et al. [2013]
Guillaume Matheron Surface reconstruction from point clouds

12. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Digne et al. [2013]
Guillaume Matheron Surface reconstruction from point clouds

13. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Finding the transport plan
"Easy" for discrete distributions using Linear Programming
(LP), but computationally intensive
Continuous distributions (faces) split into "bins"
→ (Digne et al. [2013])
Guillaume Matheron Surface reconstruction from point clouds

14. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Kantorovich-Rubinstein’s dual formulation
W1(λ, µ) = inf d(x − y)dπ(x, y) | π ∈ Π(λ, µ)
W1(λ, µ) = sup f(λ − µ) | f continuous 1-Lipschitz
→ Quickly find an approximation of the best function f.
Guillaume Matheron Surface reconstruction from point clouds

15. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Previous work
We have :
µ : discrete distribution (samples)
λ : piecewise-constant distribution (faces)
We use an analog of the Wasserstein metric :
E =
1
N
N
j=1
fjdµ −
M
i=1
wi fjdλi
2
Where :
fj is a set of chosen functions
We solve for wi
Guillaume Matheron Surface reconstruction from point clouds

16. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
E =
1
N
N
j=1
fjdµ −
M
i=1
wi fjdλi
2
rewritten as :
E =
N
j=1
bj −
M
i=1
wiaj,i
2
Minimum reached when :
∀k, 0 =
N
j=1
aj,k bj −
M
i=1
wiaj,i
→ Solution of linear system
Guillaume Matheron Surface reconstruction from point clouds

17. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Choice of fj : bounded radial functions.
aj,i = fjdλi intensive to compute → Integrable in closed
form on triangles (Hubert [2012])
Open questions :
Placement of fj : very heuristic
Problem of validation
Guillaume Matheron Surface reconstruction from point clouds

18. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Proposal
W1(λ, µ) = sup fd(λ − µ) | f continuous 1-Lipschitz
LP program in 1D (discrete) :



minimize f(1)(λ(1) − µ(1)) + · · · + f(N)(λ(N) − µ(N))
with respect to f(1), · · · f(N)
such as |f(2) − f(1)| ≤ 1
...
|f(N) − f(N − 1)| ≤ 1
(1)
Guillaume Matheron Surface reconstruction from point clouds

19. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Test case in 1D using Maple
Figure: The f function always has a slope of 1 (Lipschitz condition
attained)
Guillaume Matheron Surface reconstruction from point clouds

20. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Generalization to 2D
Project samples and triangles on an N × N lattice
LP with N2 variables
Guillaume Matheron Surface reconstruction from point clouds

21. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Visualize f
Accurate and fast
Problem : Lipschitz constraints defined in only two directions
Complexity lower than previous approach for similar resolutions
Guillaume Matheron Surface reconstruction from point clouds

22. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Wavelet Approach (Shirdhonkar and Jacobs [2008])
W1 ≈
i
|T(λ)i − T(µ)i| 2−2·j(i)
T(λ)i : i-th coefficient of the Discrete Wavelet Transform (DWT) of
λ.
→ Linear time in the number of cells ! (O(N2))
→ → w = 0.35046
λ − µ → T(λ − µ) (coefficients arranged in matrix) → Result
Guillaume Matheron Surface reconstruction from point clouds

23. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Experiments
Global minimum not where we want it to be
Problem : normalization (weight of triangle = weight of all samples
< weight of "good" samples)
Guillaume Matheron Surface reconstruction from point clouds

24. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Adjusting normalization
Problem : normalization (weight of triangle = weight of all samples
< weight of "good" samples)
W1 ≈
i
|kT(λ)i − T(µ)i| 2−2·j(i)
Best k ?
Guillaume Matheron Surface reconstruction from point clouds

25. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Convex optimization
Φ(k) =
i∈E
|kT(λ)i − T(µ)i| 2−2j(i)
Let Φi(k) = |kT(λ)i − T(µ)i|.
Figure: Φi
Guillaume Matheron Surface reconstruction from point clouds

26. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Convex optimization (cont.)
Φ is convex
Figure: Φ
O(log2(N2)) evaluations of Φ :
Complexity O(N2 log2 N)
Guillaume Matheron Surface reconstruction from point clouds

27. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Explicit enumeration
Φ(k) =
i∈E
|kT(λ)i − T(µ)i| 2−2j(i)
Let kl =
T(µ)l
T(λ)l
.
Set of singular points : k ∈ {kl | l ∈ E}.
Compute and sort all singular points (set of kl)
Find smallest Φ(kl) via dichotomy (Φ is convex)
Complexity : O(N2 log2 N)
Guillaume Matheron Surface reconstruction from point clouds

28. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Results
Fixed normalization 0.0146 (best) 0.0231
Adjusted normalization 0.0267 0.0067 (best)
Guillaume Matheron Surface reconstruction from point clouds

29. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Implementation
Wavelet transform using GSL
gsl_wavelet2d_nstransform_matrix_forward ( . . . ) ;
Multi-threading using OpenMP
#pragma omp p a r a l l e l f o r
f o r ( i n t i = 0; i < n ; i++) {
/ / ( . . . )
#pragma omp atomic
totalSimplexWeight += w;
}
LP solving using Coin|Or
Guillaume Matheron Surface reconstruction from point clouds

30. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Reconstruction
Use metric to build surface from points
Methods :
Vertex relocation
Coarse-to-fine (refinement, subdivision)
Fine-to-coarse (edge collapse, decimation)
Guillaume Matheron Surface reconstruction from point clouds

31. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Vertex relocation
Input points
Guillaume Matheron Surface reconstruction from point clouds

32. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Vertex relocation
Initialization
Guillaume Matheron Surface reconstruction from point clouds

33. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Vertex relocation
Relocating vertices along gradient of W1
Guillaume Matheron Surface reconstruction from point clouds

34. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Vertex relocation
After relocation
Guillaume Matheron Surface reconstruction from point clouds

35. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Coarse-to-fine approach
Needs guided refinement operators.
Problem : we don’t have the transport plan
Guillaume Matheron Surface reconstruction from point clouds

36. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Fine-to-coarse approach
Input points
Guillaume Matheron Surface reconstruction from point clouds

37. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Fine-to-coarse approach
Delaunay triangulation
Guillaume Matheron Surface reconstruction from point clouds

38. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Fine-to-coarse approach
Removing vertices while minimizing ∆W1
Guillaume Matheron Surface reconstruction from point clouds

39. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Fine-to-coarse approach
After decimation (5 vertices left)
Guillaume Matheron Surface reconstruction from point clouds

40. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Fine-to-coarse approach
Remove three edges, minimizing ∆W1
Guillaume Matheron Surface reconstruction from point clouds

41. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Limitations
Remove edges
→ Greedy and operator-guided for now
→ LP at each computation of W1 ?
Stopping criteria ?
Guillaume Matheron Surface reconstruction from point clouds

42. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Conclusion
Computational aspects of optimal transportation
Wavelets are fast
→ scalable in 3D
Guillaume Matheron Surface reconstruction from point clouds

43. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Future work
Local re-approximation of W1
Complexity-distortion tradeoff
Guillaume Matheron Surface reconstruction from point clouds

44. Introduction
Previous work
Dual formulation
Wavelet approach
Reconstruction
Conclusion
References
Bibliography
J. Digne, D. Cohen-Steiner, P. Alliez, F. de Goes, and M. Desbrun.
Feature-Preserving Surface Reconstruction and Simplification from
Defect-Laden Point Sets. Journal of Mathematical Imaging and
Vision, 2013.
E. Hubert. Convolution Surfaces based on Polygons for Infinite and
Compact Support Kernels. Graphical Models, 2012.
S. Shirdhonkar and D. W. Jacobs. Approximate earth mover’s
distance in linear time. 2008.
K. N. Snavely. Scene Reconstruction and Visualization from Internet
Photo Collections. PhD thesis, 2009.
Guillaume Matheron Surface reconstruction from point clouds