From noisy object surface scans to conformal unstructured grids of multiple materials for physical finite element analysis (FEA)

uib.no
U N I V E R S I T Y O F B E R G E N
From noisy object surface scans to conformal
unstructured grids of multiple materials for physical
finite element analysis (FEA)
Christian Kehl, University of Bergen / Uni Research AS
supervisor: Sophie Viseur, CEREGE/AMU

uib.no
Who am I ?
• Christian Kehl, M.Eng. cum laude (2012)
• home institution: Univ. Appl. Sciences Wismar
• internship semester A’dam (NL, 2009)
• ext. masters: Aalborg University (DK, 2011)
• MSc thesis: TU Delft (NL, 2012)
• Research: TU Deflt (NL, 2012-2014)
• Ph.D. candidacy: Uni Bergen (NO, 2014-2017)
• Ph.D. research visit: CEREGE / AMU (F, 2016/17)
• research interest: real-world scans to physical volume
models

uib.no
Motivation
• 3D data acquisition gets easier, cheaper and more
accessible
• Range of equipment allows acquisition for any budget,
adapted to quality demands
• volumetric analysis of the physical world at the core of
many science disciplines
– medicine and biomechanics; geology (nat. res.);
archaeology; planetary studies; climate change;
natural disasters; urban heat management;
mechanics and materials; aerospace engineering;
energy science;

uib.no
Motivation
• great potential and interest in simulations vs. geometric
knowledge of domain experts
• Requirements and constraints:
– high simulation accuracy
– large data (extent & resolution; dimensionality and
time-dependency)
– limited computational resources (field studies;
desktop simulations)
– budget ...
• transfer geometric knowledge into tangible software &
algorithms

uib.no
Point Set Scanning: LiDAR
• issue: any material
(gas) between scanner
and object attenuates I
• problems:
– scattered reflection
(metals)
– refracting materials
(gems, fluids, glass ...)
• noise: attenuation,
scattering, refracted
light
• uniform density,
irregular positioning

uib.no
Point Set Scanning: Structured Light
structured pattern can
also be projected in
non-visible wavelengths
(=> Xbox Kinect)
• range imaging
(disparity)
• projection of structured
pattern
• problems:
– scattered reflection
(metals)
– refracting materials
(gems, fluids, glass ...)
– wave interferences
• uniform density; regular
lattice positioning

uib.no
Point Set Scanning: SfM
• relies on:
– image quality
– # images; temporal
correlation
– lighting & materials (light
scattering, specular
highlights, ...)
– point correlation acc.
– numerical optimisation
algorithm
• flat, blank object (areas) 
no “features”  scan holes
• advantage: underwater
scan
• non-uniform density;
irregular positioning

uib.no
Examples: Point Set Scanning
LiDAR
Structure-from-
Motionstructured light

uib.no
Point Set Scanning
• LiDAR: the accuracy references; issue: $$$
• structured light: cheap alternative indoors; outdoors
(Kinect) -> IR interference: the sun
• Structure-from-Motion (SfM): versatile & cheap;
demands knowledge & patience
• Alternative: Stereo Imaging
• More info: ISPRS & Photogrammetric Record

uib.no
Mono-material reconstruction
• geometric demands:
– coherently-oriented surface elements
– hole-free
– topologically correct shape reconstruction
– density adaptive (optional, but advantageous)
– closed C2 surface (i.e. tight envelope)
– non-manifold (for Delaunay Tetrahedralisation)
– high-quality volume elements
– minimal volume element count (simulation
convergence)

uib.no
• Problem: irregular, noisy scans vs. exact-geometry
reconstruction
• Exact-geometry schemes often fail:
– Delaunay Triangulation (e.g. Shewchuck 1996, Alliez
et al. 2011)
– Cocone (Dey & Goswami 2003, Dey and Levine 2009)
– PowerCrust (Amenta et al. 2001)
– Alpha Shapes (Edelsbrunner & Mücke 1992)
– Ball Pivoting (Bernardini et al. 1999)
• varying point density & holes  insufficient samples for
surface reconstruction

uib.no
• RMLS (Fleishman et al. 2005) and Poisson surfaces
(Khazdan et al. 2006) account for varying sample density
• Tetrahedralisation for FEA without surface geometry
changes possible [George et al. 1991]
• persisting issues:
– surface: self-intersection, manifolds, triangle count
– overly smooth surface approximation; crease angles
– new vertex set instead of triangulating the original
– lack of theoretical guarantees of shape approximation

uib.no

uib.no
• specific application: estimate mineral volumes
– current volume estimation means very expensive
– currently lab experimental estimation  optical scans
as cheap & less labour-intense alternative

uib.no
Segmentation of surfaces
• Until now:
– scan + surface & volume reconstruction of 1 object
– treating as homogeneous surface / material / object
• Physical reality:
– object consists of sub-entities
– entities are heterogeneous (from one another as
potentially in itself)
 multiple materials
 labelling, semantics and segmentation

uib.no
• Goal: segment a given (surface) geometry into distinct
regions, based on its intrinsic properties
• specific application case/ motivation for our studies:
Interactive segmentation of outcrop surfaces into its
composing elements, on mobile devices
• PhD research: “Visual Techniques for Geological
Fieldwork Using Mobile Devices” [Kehl et etl. 2015/2016]

uib.no
• application:
segmentation outdoors,
on tablets
• conditions:
– domain expert influence
– fast computation, limited
hardware performance
– input: anisotropic,
irregular surface meshes
– no change of underlying
surface structure
– noise-resilient

uib.no
• algorithmic space: parametric vs. geometric
• general segmentation:
– part-aware [Liu et al. 2009]
– geometric intrinsic (e.g. curvature-based [van Kaick
et al. 2014])
– interactive ([Zhang et al. 2011])
• Good reviews: Shamir 2008, Benhabiles et al. 2010,
Theologou et al. 2015
• Our approach: interactive; combine geometry,
morphology & statistical optimisation

uib.no
• Key algorithmic components:
– combinatorial expansion, by curvature & extrema
– morphology operations (erode,dilate,open,close)
– stochastic optimisation  simulated annealing (SA)
• alg. components are known, but morphology and SA on
unstructured, irregular meshes undefined
principal shapes for geometric
classification [Kudelski et al. 2011]
morphological classification, including topo-
logical guarantees [Williams & Rossignac 2004]
stochastic active contours by
simulated annealing [Horritt 1999]

uib.no
Segmentation of surfaces – combinatorics & curvature
• Starting point: Interactive initialisation via lines on
surface
• surface integration: flag by line intersection

uib.no
• Next: expand initial area to boundary curve
• boundary conditions: direction- weighted geometry-
intrinsic (i.e. curvature, extrema)
• iterative refinement:
• track boundary vertices
• check inside-validity criterion
• include vertex
• make its neighbours boundary
candidates
• repeat until equilibrium





 














else
nnvNvvvkvk
F
else
vNvvkvk
vk
F
bnnnn
bnnifl
:0
1),(),())(sgn())(sgn(:1
:cond.changeSign
:0
)())(sgn())(sgn(:1
0)(:1
:cond.pointInfliction
12
sgn
2121
    0)(1)()0)((, sgn  vFvFvFVvSv iflifl

uib.no
Segmentation of surfaces: process

uib.no
• EXAMPLE 1:
EXPANSION WITH
ISLE
• EXAMPLE 2:
BASELINE

uib.no
• Pro:
– versatile boundary condition
– VERY FAST optimisation compared to alternatives
(e.g. particle systems, active contours)
• Contra:
– isle occurrences
– thin bridges
– sharp (i.e. zig-zag) contour for irregular meshes

uib.no
• Pro:
– versatile boundary condition
– VERY FAST optimisation compared to alternatives
(e.g. particle systems, active contours)
• Contra:
– isle occurrences
– thin bridges
– sharp (i.e. zig-zag) contour for irregular meshes
=> MORPHOLOGY
=> CURVE STRAIN ENERGY
=> FILTERING

uib.no
Segmentation of surfaces - morphology
• Morphology, Curve Strain Energy and Filtering
Tightening [Williams & Rossignac 2005]
• mapping to discrete space: pixel cluster = triangle
StrtCStSDilate s  
,|:
StrtCStSErode s  
,|:
SSRSSRSR rrrrr  
)(,)(:)(Closing
SSFSSFSF rrrrr  
)(,)(:)(Opening
 )(,,)(Anticore
)()(Mortar
)()(Core
SMStTtTSA
SFSM
SRSX
AAAA
r
r




uib.no
Segmentation of surfaces – simulated annealing opt.
• Williams & Rossignac 2005: Fast Marching curve
evolution, curvature speed func. [Osher & Sethian 1988]
• Fast Marching in discrete space undefined => stat.
optimisation via simulated annealing (SA)
• advantage: morphology defines topology => no checks
• Open problems to address:
– r-constrain on the surface
– convexity-based energy function
     




























M
j
K
k
inouti
i
i
kjicjicc
ceSE
ceSE
0 0
1
1
1
,,,
1)'(
1)(
:functionsEnergy


 
]2.1,8.0[
:0
)()'(:
1
),'(),(
:yprobabilitAcceptance
1

























else
SESE
c
e
SESEP i

uib.no

uib.no
SA
optimisation;
morph.
r-constrain
0.10
Input
SA
optimisation;
morph.
r-constrain
0.13
Output

uib.no
Multi-material volume reconstruction
• Given a segmented volume image, how do we construct
minimal, accurate, conformal FEM meshes?
• Known approaches:
– (weighted) Delaunay based on interfaces [Boltcheva
et al. 2009[a]]
– Lattice Cleaving [Bronson et al. 2014]
– BioMesh3D [Meyer et al. 2007/2008]
=> starting point

uib.no
• Specific application scenario: patient-specific prostheses
– specific case: femur replacement
– patient-specific prosthesis design leads to less
complications after operation
– adapting design to “normal stress” of the patient:
accurate FEA and FEA models
– details:
• MSc Thesis Christian Kehl (2012) “Conformal multi-
material mesh generation from labelled medical
volumes”
• PhD Thesis Daniel F. Malan (2015) “Pinning down
loosened prostheses: imaging and planning of
percutaneous hip reﬁxation”

uib.no
orthopaedic workflow hip replacement: CT scanning of the patient (a); segmenting
scan into regions of different objects (semantics), respectively: different materials (b);
constructing high-quality, minimal-element FEM volume mesh (c); FEA stress
simulation in bones (d);
FINAL part (not depicted): refine prosthesis (central, grey, ‘banana-shaped’ object)
design and positioning based on stress simulation; repair or insert prosthesis.

uib.no
• conformal meshes: meshes sharing vertex- and edge
configurations on their interfaces
• minimal mesh: describing an object’s shape / volume
with a minimal set geometric primitives accurately
=> inter-vertex distances of interface surfaces can vary
significantly

uib.no
• minimal inter-vertex distance => minimum radius of
curvature (rmin) expressed in the local feature size (λ)
• sampling theorem: ε-sampling [Amenta et al. 1998[b],
Boissonnat and Oudot 2005, Meyer et al. 2005,
Shewchuck 2008]

uib.no
• needs: 3D skeleton / medial axis transform
• approximation; finding the minimum medial axis is NP-
hard [Coeurjolly et al. 2008]
• approach Meyer et al. 2007: high-density proxy surfaces
at interfaces
• MAT from implicit representation of proxy surfaces (see
[Hesselink & Roerdink 2008, Coeurjolly & Montanvert
2007])

uib.no
• next: construct distance field: surface  MAT
• original, segmented voxel centres (as vertices, vo) are
“on” or “close-to” the proxy surface (Tp)
0 < d(vo,Tp) < λ(Tp)
• sample λ(vo) from the distance field
• result: maximal distance at vo to guarantee distance to
adjacent vertices

uib.no
• Next: Particle system [Witkin & Heckbert 1994, Meyer et
al. 2005]; voxel centres are particles (seeds)
• move particles on proxy surface: max. inter-particle
spacing, min. energy:
speedparticlematidentIgradientfuncimplFposparticle vp ii


.;.;...;
E = energy function to minimize
(functions from Meyer et al. 2005)

uib.no
Seed particles with surface normals and
distance constraint (arrow colour), moving
on proxy surface
segmented seed
reconstructed
surface & MAT
vert.

uib.no
• Final step: construct final surface(s) [per material] from
optimised particle set via Delaunay Triangulation
=> mathematically well defined iff particle satisfies ε-
criterion [Meyer et al. 2007, Amenta et al. 1998[b]]
• preserve segmentation / material label via initial,
segmented volume image

uib.no
• Issue: ε-criterion (for Delaunay) violated in domains of
crease angles
• Our extension: meshing by injection
– optimise vertex positions via particle system
– iteratively inject particles in proxy mesh
– remove original proxy vertices iteratively
– enforces edge consistency by edge flipping
• algorithm for crease-angle domains and sparse samples,
as not bound by ε-criterion

uib.no
• publication of the algorithm currently on halt; expand and
resubmit end 2017/ begin 2018
initial workshop paper project website
Meyer et al., 2008,
high sample
Meyer et al., 2008 Kehl et al., 2015

uib.no
Future Vision - Conceptual
• Multi-material meshing: extension to segmented point
set meshing
– combine volume meshing from optical scans with
multi-material meshing
– proxy surfaces also derivable for point sets
(theoretically)
– particle system sampling optimises final vertex
positions and drastically reduces tetrahedral element
count
– address some interesting applications ...

uib.no
Future Vision - Conceptual
• Multi-material meshing: influence of surface sample
schemes
– particle optimisation computationally very expensive;
demands proxy surface
– with noisy point sets from scanning: disadvantageous
– great study on different vertex samples: Pilleboue et
al. 2015
– idea:
• evaluate given noisy sample for suitable sampling
scheme representative
• chose reconstruction method based on given sample

uib.no
Future Vision- Applications
• Volume Reconstruction in geosciences (lab)
– currently being done: volume mesh from sketched
surfaces (Rapid Reservoir Modelling;
www.rapidreservoir.org; Jacquemyn et al. 2017)
– goal: seamlessly integrate natural observation (digital
outcrop) in conceptual models (e.g. Caumon et al.
2004, Jackson et al. 2015,)
– physical stress simulation, directly on outcrop scan

uib.no
Future Vision - Applications
• Volume Reconstruction in geosciences (field)
– scanning hand samples in the field
– integrate volume & simulation result back on field
tablet
– approximate stress/cleavage simulations on small-
scale samples on the tablet via GPU Computing

uib.no
References
[Alliez 2011] P. Alliez, L. Saboret & G. Guennebaud, “Surface Reconstruction from Point Sets”, 2001
[Amenta1998b] N. Amenta, M. Bern & D. Eppstein, “The crust and the beta-Skeleton: combinatorial curve reconstruction”, Graph. Models and
Image Proc., vol. 60, p. 125-135, 1998
[Amenta2001] N.Amenta, S. Choi & R.K. Kolluri, “The Power Crust”, 6th Symposium on Solid Modelling and Applications, p. 249-266, 2001
[Benhabiles2010] H. Benhabiles, J.-P. Vandeborre, G. Lavoué & M. Daoudi, “A comparitive study of existing metrics for 3D-mesh
segmentation evaluation”, The Visual Computer, vol. 26 no. 12, p. 1451-1466, 2010
[Bernardini2010] F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin, “The Ball-Pivoting Algorithm for Surface Reconstruction”,
IEEE Trans. Vis. Comp. Graph., vol. 5, p. 349-359, 1999
[Boissonnat2005] J.-D. Boissonnat & S. Oudot, “Provably good sampling and meshing of surfaces”, Graphical Models, vol. 67 no. 5, p. 405-451,
2005
[Boltcheva2009] D. Boltcheva, M. Yvinec & J.-D. Boissonnat, “Feature preserving Delaunay mesh generation from 3D multi-material images”,
Computer Graphics Forum, vol. 28 no. 5, 2009
[Bronson2014] J. Bronson, J.A. Levine & R.T. Whitaker, “Lattice Cleaving: A Multimaterial Tetrahedral Meshing Algorithm with Guarantees”,
IEEE Trans. Vis. Comp. Graph., vol. 20 no. 2, p. 223-237, 2014
[Coeurjolly2008] D. Coeurjolly, J. Hulin & I. Sivignon, “Finding a minimum medial axis of a discrete shape is NP-hard”, Theoretical Comp. Sci.,
vol. 406 no. 1, p. 72-79, 2008
[Coeurjolly2007] D. Coeurjolly & A. Montanvert, “Optimal Separable Algorithms to Compute the Reverse Euclidean Distance Transformation and
Discrete Medial Axis in Arbitrary Dimension”, IEEE PAMI, vol. 29 no. 3, p. 437-448, 2007
[Dey2003] T.K. Dey & S. Goswami, “Tight Cocone: A water tight surface reconstuctor”, 8th Symposium on Solid Modelling and
Applications, 2003
[Dey2009] T.K. Dey & J.A. Levine, “Delaunay Meshing of Piecewise Smooth Complexes without Expensive Predicates”, J. Algorithms, vol.
2 no. 4, 1327-1349, 2009
[Edelsbrunner1992] H. Edelsbrunner & E.P. Mücke, “Three-dimensional alpha shapes”, Proc. ACM Volume Visualization, p. 75-82, 1992
[Fleishman2005] S. Fleishman, D. Cohen-Or & C. Silva, “Robust moving least-squares fitting with sharp features”, ACM TOG, vol. 24, p. 544-
552, 2005
[George1991] P.L. George, F. Hecht & E. Saltel, “Automatic mesh generator with specified boundary”, Computer Methods in Applied
Mechanics and Engineering, vol. 92 no. 3, p. 269-288, 1991
[Hesselink2008] W.H. Hesselink & J.B.T.M. Roerdink, “Euclidean Skeletons of Digital Image and Volume Data in Linear Time by the Integer
Medial Axis Transform”, IEEE PAMI, vol. 30 no 12, p. 2204-2217, 2008
[Jackson2015] M.D. Jackson, J. Percival, P. Mostaghimi, B. Tollit, D. Pavlidis, C. Pain, J. Gomes, A.H. Elsheikh, P. Salinas & A. Muggeridge,
“Reservoir modeling for flow simulation by use of surfaces, adaptive unstructured meshes, and an overlapping-control-volume
finite-element method”, SPE Reservoir Evaluation & Engineering, vol. 18 no. 2, p. 115-132, 2015

uib.no
References
[Jacquemyn2017] C. Jacquemyn, Y. Melnikova, M.D. Jackson & G.J. Hampson, “Surface-Based Modelling of Subsurface Reservoirs Using
Parametric NURBS Surfaces”, 79th EAGE Conference and Exhibition, 2017
[VanKaick2014] O. van Kaick, N. Fish, Y. Kleiman, S. Asafi & D. Cohen-Or, “Shape Segmentation by Approximate Convexity Analysis”, ACM
TOG, vol. 34 no. 4, p. 1-11, 2014
[Kazhdan2006] M. Kazhdan, M. Bolitho & H. Hoppe, “Poisson surface reconstruction”, Proc. Symp. Geometry Processing, p. 61-70, 2006
[Kehl2016] C. Kehl, S. J. Buckley, R. L. Gawthorpe, I. Viola & J. A. Howell, “DIRECT IMAGE-TO-GEOMETRY REGISTRATION USING
MOBILE SENSOR DATA”, ISPRS Annals, 2016
[Kehl2017] C. Kehl, S.J. Buckley, S. Viseur, R.L. Gawthorpe & J.A. Howell, “Automatic Illumination-Invariant Image-to-Geometry
Registration in Outdoor Environments”, The Photogrammetric Record, vol. 32 no. 158, 2017
[Kudelski2011] D. Kudelski, S. Viseur, G. Scrofani & J.L. Mari, “FEATURE LINE EXTRACTION ON MESHES THROUGH VERTEX MARKING
AND 2D TOPOLOGICAL OPERATORS”, Int. J. Image and Graphics, vol. 11 no. 4, p. 531-548, 2011
[Liu2009] R. Liu, H. Zhang, A. Shamir & D. Cohen-Or, “A Part-aware Surface Metric for Shape Analysis”, Computer Graphics Forum, vol.
28 no. 2, p. 397-406, 2009
[Meyer2005] M.D. Meyer, P. Georgel & R.T. Whitaker, “Robust particle systems for curvature dependent sampling of implicit surfaces”, Proc.
Shape Modelling and Appl., 2005
[Meyer2007] M.D. Meyer, R.M. Kirby & R.T. Whitaker, “Topology, Accuracy, and Quality of Isosurface Meshes Using Dynamic Particles”,
IEEE Trans. Vis. Comp. Graph., vol. 13 no. 6, p. 1704-1711, 2007
[Meyer2008] M.D. Meyer, R.T. Whitaker, R.M. Kirby, C. Ledergerber & H.-P. Pfister, “Particle-based Sampling and Meshing of Surfaces in
Multimaterial Volumes”, IEEE Trans. Vis. Comp. Graph., vol. 14 no. 6, p. 1539-1546, 2008
[Osher1988] S. Osher & J.A. Sethian, “Fronts Propagating with Curvature Dependent Speed: Algorithms Based on Hamilton-Jacobi
Formulations”, J. Comp. Physics, vol. 79, p. 12-49, 1988
[Pilleboue2015] A. Pilleboue, G. Singh, D. Coeurjolly, M. Kazhdan & V. Ostromoukhov, “Variance Analysis for Monte Carlo Integration”, ACM
TOG, vol. 34 no. 4, 2015
[Shamir2008] A. Shamir, “A survey on Mesh Segmentation Techniques”, Computer Graphics Forum, vol. 26 no. 6, p. 1539-1556, 2008
[Shewchuk1996] J.R. Shewchuk, “Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator”, LNCS Applied Computational
Geometry, p. 203-222, 1996
[Shewchuk2008] J.R. Shewchuk, “General-dimensional constrained Delaunay and constrained regular triangulations, I: Combinatorial
properties”, Discrete & Computational Geometry, vol. 39, no. 1, p. 580-637, 2008
[Shewchuk2014] J.R. Shewchuk & H. Si, “Higher-Quality Tetrahedral Mesh Generation for Domains with Small Angles by Constrained Delaunay
Refinement”, Proc. 13th Symp. Computational Geometry, p. 290-299, 2014
{Theologou2015] P. Theologou and I. Pratikakis and T. Theoharis, “A comprehensive overview of methodologies and performance evaluation
frameworks in 3D mesh segmentation “, Computer Vision and Understanding, vol- 135, p. 49-82, 2015
[Williams2005] J. Williams & J. Rossignac, “Tightening: Curvature-limiting morphological simplification”, 6th Symposium on Solid and Physical
Modelling, 2005
[Witkin1994] A. Wikin and P.S. Heckbert, “Using Particles to Sample and Control Implicit Surfaces”, 1984

From noisy object surface scans to conformal unstructured grids of multiple materials for physical finite element analysis (FEA)

Recommended

Recommended

More Related Content

Similar to From noisy object surface scans to conformal unstructured grids of multiple materials for physical finite element analysis (FEA)

Similar to From noisy object surface scans to conformal unstructured grids of multiple materials for physical finite element analysis (FEA) (20)

More from Christian Kehl

More from Christian Kehl (20)

Recently uploaded

Recently uploaded (20)

From noisy object surface scans to conformal unstructured grids of multiple materials for physical finite element analysis (FEA)