Point Cloud Processing: Estimating Normal Vectors and Curvature Indicators using Eigenvectors

1Challenge the future
Basic Point Cloud Processing
Estimating Normal Vectors and Curvature Indicators
Ir. Pirouz Nourian
PhD candidate & Instructor, chair of Design Informatics, since 2010
MSc in Architecture 2009
BSc in Control Engineering 2005
Geo1004, Geomatics Master Track
Directed by Dr. ir. Sisi Zlatannova

How do we make a computer ‘see’ what we understand from this?
What do you make of this?
What is this all about?

Data to Information: LIDAR to LOD2
How do we make sense out of such big data?
Image courtesy of GEOCONNEXION
?

SOFTWARE APPLICATIONS
• FME, Safe Software
• LASTools, rapidlasso
• Point Cloud Library (PCL)
• Cloud Compare
• MeshLab
• LP360 QCoherent
A NEW VERSION OF TOIDAR! YES!
http://opentopo.sdsc.eduInformation from:

FME, Safe Software
transform, convert, translate, extract, integrate, automated, repeatable

LASTools, rapidlasso
filtering, clipping, reprojecting, compression, classification, DSM, DTM, TIN, contours bare-earth
de facto standards for
point cloud data:
.las
.laz

LP360, QCoherenet
LIDAR, Classification, Breaklines, Visualization, Extraction, Automatic

Point Cloud Library PCL
point clouds, visualization, processing, segmentation, filtering,
feature estimation, registration
Using this library in Rhino?

Cloud Compare
Implements PCL and more methods, handy to use for point
cloud processing
Image from software.informer.com

MeshLab
Has some surface reconstruction methods, handy to have when
working with point clouds.
Image from https://danieleferdani.wordpress.com/

Brainstorm
A sample building extracted from the AHN2 dataset.
?
?

Part 1 finished
We will continue with one specific way of dealing with point
clouds and estimating normal vectors and curvatures…

An idea!
A sample building extracted from the AHN2 dataset.

A Curvature-Based Approach to Point Cloud
Segmentation & Feature Detection
Removing
Outliers
Elevation
Classification
Slope
Classification
Aspect
Classification
Region-Growing
Segmentation
Point Cloud
of a Building
Noise
reduction
Forming
Neighbourhoods:
e.g. KNN or Range
Forming
Covariance
Matrices for
Neighbourhoods
Using PCA, Estimating:
Normal Vectors &
Curvature Indicators
Normal
Vectors
Curvature
Indicators
Eigen
System
Triangulation
Mesh
Surface

Different Approaches for Finding Fitting Planes/Edges
1. Using curvature values computed (estimated) by eigenvalues of
covariance matrices, to run a segmentation algorithm based on region
growing.
2. Using Octree Voxels, to detect edge voxels, remove them and create
segments.
3. Using Hough transform on a 2.5D point cloud, converting to the
parameter space and using DBScan clustering method to find
clusters of parameters, each of which correspond to a segment in a
point cloud.
4. And a few more in the literature…

Different Approaches for Finding Fitting Planes/Edges
Region growing based on
normals and curvature Segmentation usingOctree voxels Hough transformation

Different Approaches for Finding Fitting Planes/Edges:
our experience!
Curvature based region
growing
Voxel based region
growing
Hough transform
Noise management Fair Good Good
Density variation
management
Fair Fair Good
Efficiency Good Good Poor
Ambiguity of vantage
point management
Fair Good Good
Avoiding priori
knowledge
Good Good Poor
Ease of further
processing
Good Poor Poor

Neighbourhoods of Points
Fixed Distance Neighbors (FDN) and K-Nearest Neighbors (KNN)
• Rabbani, T., van den Heuvel, F., & Vosselmann, G. (2006). Segmentation of point clouds using smoothness
constraint. International Archives of Photogrammetry, Remote Sensing and Spatial Information
Sciences, 36(5), 248-253.
• Pauling, Frederick, Michael Bosse, and Robert Zlot. "Automatic segmentation of 3d laser point clouds by
ellipsoidal region growing." Australasian Conference on Robotics and Automation. 2009.
The idea is to form neighborhoods based on [squared] distance
These two options are usually provided. Apart from normal vector
estimation KNN can be useful also for removal of outliers.

Fitting Planes to Neighbourhoods
The idea is that the underlying surface is a 2-manifold; therefore it
resembles a 2D plane locally
Least Square Plane Fitting Estimation Problem
 Principal Component Analysis Problem
Intuition: Defining plane as the locus of lines that have
direction vectors perpendicular to a normal vector;
considering an origin for the plane in questions, we can
consider the plane as the locus of points A such that A-
O is perpendicular to N.

resembles a 2D plane locally, we look at ellipsoids showing local
dispersions
Images courtesy of Olga Sorkine

How to estimate normals using PCA
resembles a 2D plane locally
• Pauly, Mark, Markus Gross, and Leif P. Kobbelt. "Efficient simplification of
point-sampled surfaces." Proceedings of the conference on
Visualization'02. IEEE Computer Society, 2002.
• Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., Stuetzle, W. Surface
reconstruction from unorganized points. SIGGRAPH 92, 1992
• Shaffer, E., Garland, M. Efficient Adaptive Simplification of Massive
Meshes. IEEE Visualization 01, 2001

We form a covariance matrix for each neighborhood, showing how
neighbors are dispersed around their average (centroid).
This will be a 3
by 3 symmetric
matrix!

We form a covariance matrix for each neighborhood, showing how
neighbors are dispersed around their average (centroid).
This will be a 3
by 3 symmetric
matrix!
Form an Eigen system for this matrix using a
linear algebra library, and the first eigenvector
corresponding to least eigenvalue will be the
normal vector at each neighbourhood.
Explanation follows…
PCL implementation:
http://pointclouds.org/documentation/tutorials/normal_estimation.php

How to estimate curvature using PCA
The idea is to use an indication of change along the normal vector
Jolliffe, I. Principle Component Analysis. Springer-Verlag, 1986

How to do all this in code?
• We try not to reinvent the wheel; the idea is to use free open
source libraries like Math.NET and Accord.NET
• We are not the first people dealing with such issues, these are
generally matters of data mining and machine learning
• We can find KNN neighbourhoods using
Accord.NET http://accord-framework.net/
• We can find eigenvalues and eigenvectors
using MetaNumerics.dll, MathNet.dll or
Accord.NET
• You will receive example code implementing
MetaNumerics

How do we minimize the error in fitting a plane to the
mentioned neighbourhood? First we define it…
Images courtesy of Olga Sorkine

An explanation after Olga Sorkine:
• Input points:
• Centroid:
• Vectors from the centroid:
m

Continued…
mm minimizes SSD and it can
be shown that m is the
centroid C
We can rewrite the problem as:

Continued…
m
m minimizes SSD, formally it is the
“arg min” of the following function
(to be minimized), i.e. the argument
that makes it reach its minimum. and
it can be shown that m is the
centroid C
Solving this turns out to be equal to
solving an Eigen system for the
covariance matrix; you can see how…

Long story short…
We find eigenvalues and eigenvectors of the covariance matrices…

Long story short…
We find eigenvalues and eigenvectors of the covariance matrices…
What was this all about? Segmenting the point cloud taking into
account [underlying] surface variations, in search of smooth patches,
made disjoint by edges (where we find high curvature/variation).
Therefore we can use the above estimated vectors and values for
such a segmentation.

Explanation from lecture notes of Olga Sorkine…

Continued from lecture notes of Olga Sorkine…

This S is our covariance
matrix, remember?

•Constrained minimization – Lagrange multipliers
Given this equality in
differentiation of vector
valued functions

maximize f(x, y) subject to g(x, y) = c.
Lagrangian:
The prblem is transformed to finding a ‘’staionary
point for the Langrangian at which the partial
derivatives of Lagrangian function are zero.

This one means that the
normal vector is
normalized, meaning its
length is equal to 1; why,
because the dot product
of a vector by itself gives
the squared length

How to form covariance matrices in code?
VB.NET (confirm and debug for yourself)
If(Pts.Count > 3) Then
Dim CovMatrices As New list(Of Meta.Numerics.Matrices.SymmetricMatrix)
For j As Integer=0 To Pts.Count - 1
Dim CovM As New Meta.Numerics.Matrices.SymmetricMatrix(3)
Dim Neighbors As List(Of Integer) = DirectCast(Neigh(j), List(Of Integer))
Dim Centroid As New Point3d
Dim NPts As New List(Of Point3d)
For Each neighbor As Integer In Neighbors
Centroid = Centroid + Pts(neighbor)
NPts.Add(Pts(neighbor))
Next
Centroid = Centroid / Neighbors.Count
Dim CiCBar As New RectangularMatrix(3, Neighbors.Count)
For k As Integer=0 To Neighbors.count - 1
Dim Diff As point3d = Pts(Neighbors(k)) - Centroid
CiCBar(0, k) = Diff.X
CiCBar(1, k) = Diff.y
CiCBar(2, k) = Diff.z
Next
CovM = CiCBar.MultiplySelfByTranspose()
CovM = (1 / (Neighbors.count - 1)) * CovM
CovMatrices.Add(CovM)
Next
A = CovMatrices(0).ToArray()
C = CovMatrices
End If

How to form covariance matrices in code?
C#.NET (confirm and debug for yourself)
if ((Pts.Count > 3)) {
List<Meta.Numerics.Matrices.SymmetricMatrix> CovMatrices = new List<Meta.Numerics.Matrices.SymmetricMatrix>();
for (int j = 0; j <= Pts.Count - 1; j++) {
Meta.Numerics.Matrices.SymmetricMatrix CovM = new Meta.Numerics.Matrices.SymmetricMatrix(3);
List<int> Neighbors = (List<int>)Neigh(j);
Point3d Centroid = new Point3d();
List<Point3d> NPts = new List<Point3d>();
foreach (int neighbor in Neighbors) {
Centroid = Centroid + Pts(neighbor);
NPts.Add(Pts(neighbor));
}
Centroid = Centroid / Neighbors.Count;
RectangularMatrix CiCBar = new RectangularMatrix(3, Neighbors.Count);
for (int k = 0; k <= Neighbors.count - 1; k++) {
point3d Diff = Pts(Neighbors(k)) - Centroid;
CiCBar(0, k) = Diff.X;
CiCBar(1, k) = Diff.y;
CiCBar(2, k) = Diff.z;
}
CovM = CiCBar.MultiplySelfByTranspose();
CovM = (1 / (Neighbors.count - 1)) * CovM;
CovMatrices.Add(CovM);
}
A = CovMatrices(0).ToArray();
C = CovMatrices;
}

Part 2 finished
Some hints on previous assignment follow…

Grid Surface Reconstruction
Pseudo-Code
Define a new Mesh with a list of Vertices and a list of Faces
Mesh.Vertices=Points
For j as Integer=0 to V-2//choose only bottom-left corners as pivot points
For i As Integer=0 To U - 2
define n0,n1,n2,n3 As Integer
n0 = j * U + I //bottom-left
n1 = n0 + 1 //bottom-right
n2 = n0 + U //top-right
n3 = n1 + U //top-left
Define face As new MeshFace(n0, n1, n3, n2)
Mesh.Faces.AddFace(face)
Next
Next

VB.NET
Dim M As New Mesh
M.Vertices.AddVertices(P)
If Not C Is Nothing Then
M.VertexColors.AppendColors(C.toArray)
For j As Integer=0 To V - 2
For i As Integer=0 To U - 2
Dim n0,n1,n2,n3 As Integer
n0 = j * U + i
n1 = n0 + 1
n2 = n0 + U
n3 = n1 + U
Dim face As MeshFace = New MeshFace(n0, n1, n3, n2)
M.Faces.AddFace(face)
Next
Next
with two nested loops

VB.NET
Dim M As New Mesh
M.Vertices.AddVertices(P)
If Not C Is Nothing Then
M.VertexColors.AppendColors(C.toArray)
For i As Integer=0 To U * (V - 1) - 1
If (i Mod u) < u - 1 Then
Dim n0,n1,n2,n3 As Integer
n0 = i
n1 = n0 + 1
n2 = n0 + U
n3 = n1 + U
Dim face As MeshFace = New MeshFace(n0, n1, n3, n2)
M.Faces.AddFace(face)
End If
Next
with one loop

C#.NET
Mesh M = new Mesh();
M.Vertices.AddVertices(P);
if ((C != null))
M.VertexColors.AppendColors(C.toArray);
for (int i = 0; i <= U * (V - 1) - 1; i++) {
if ((i % u) < u - 1) {
int n0 = 0;
int n1 = 0;
int n2 = 0;
int n3 = 0;
n0 = i;
n1 = n0 + 1;
n2 = n0 + U;
n3 = n1 + U;
MeshFace face = new MeshFace(n0,
n1, n3, n2);
M.Faces.AddFace(face);
}
}
with one loop

Simple Estimation of Normal
Vectors Pseudo-Code
Form an empty list of normal vectors
Define deviation as a double
For each point as Point3d in the point cloud
find neighbors
fit a plane to neighbors
Get the normal of this plane and put it out as the normal of the point
form a vector from the vantage point VP to point=VP-point and call it dir
if this normal.dir>0 then
Add the normal to the list of normals
Else
Add –normal to the list of normals
End
Next

Estimation Normal Vectors
C#.NET
List<Vector3d> Normals = new List<Vector3d>();
Point3dList PCList = new Point3dList();
PCList.AddRange(x);
double Dev = MD;
foreach (Point3d point in PCList) {
dynamic Neighbors = PCList.FindAll(V => V.DistanceTo(point) < D);
plane NP = default(Plane);
Plane.FitPlaneToPoints(Neighbors, NP, Dev);
if (NP.Normal * (VP - point) > 0) {
Normals.Add(NP.Normal);
} else {
Normals.Add(-NP.Normal);
}
}
A = Normals;
B = PCList.FindAll(VT => VT.DistanceTo(x(654)) < D);

Estimation Normal Vectors
VB.NET
Dim Normals As New list(Of Vector3d)
Dim PCList As New point3dlist
PClist.AddRange(x)
Dim Dev As Double = MD
For Each point As point3d In PClist
Dim Neighbors = PClist.FindAll(Function(V) V.distanceto(point) < D)
Dim NP As plane
Plane.FitPlaneToPoints(Neighbors, NP, Dev)
If NP.Normal * (VP - point) > 0 Then
Normals.Add(NP.Normal)
Else
Normals.Add(-NP.Normal)
End if
Next
A = Normals
B = PClist.FindAll(Function(VT) VT.DistanceTo(x(654)) < D)

Basics of Scripting in Rhino + GH
• VB.NET (Sub[routine], Function, ByVal, ByRef)
• C#.NET (Void, [Functions], [val],ref)
• Python (defs)
Basic concepts of systems and modules, Inputs & Outputs
Private Sub RunScript(ByVal x As Object, ByVal y As Object, ByRef A As
Object)
End Sub
'<Custom additional code>
'</Custom additional code>
Function plus(A As Integer, B As Integer) As
Integer
Return A + B
End Function
private void RunScript(object x, object y, ref object A)
{
}
// <Custom additional code>
// </Custom additional code>
public int plus(int a, int b)
{
return a + b;
}
def plus(a,b):
return a+b
Nothing visible.

Basics of Scripting in Rhino + GH
• Rhinocommon
• Rhinoscript
• Grasshopper Kernel
Rhinocommon
Library of
Geometry
Operations
VB.NET. C#.NET &
Python
Grasshopper Kernel
Library of Some
Special Geometry
Operations
VB.NET. C#.NET &
Python
Rhionscript (with [Iron]Python)
Operations as in Rhino Command Line
Python
http://4.rhino3d.com/5/rhinocommon/Rhinocommon SDK:
Grasshopper SDK: [Rhino]Command: GrasshopperGetSDKDocumentation
Rhinoscript Syntax SDK: [GH Python]Help: Rhinoscript syntax help

Questions? p.nourian@tudelft.nl
• I will give you the tools we have developed so far as open
source;
• You will make them better using the following libraries;
• Using either Math.NET or MetaNumerics is a must;
• Using Accord.NET is a big plus!
• If you are using Python, use similar libraries such as SciPy
• If you want to do something else let us discuss it now!

Point Cloud Processing: Estimating Normal Vectors and Curvature Indicators using Eigenvectors

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (7)

Similar to Point Cloud Processing: Estimating Normal Vectors and Curvature Indicators using Eigenvectors

Similar to Point Cloud Processing: Estimating Normal Vectors and Curvature Indicators using Eigenvectors (20)

More from Pirouz Nourian

More from Pirouz Nourian (8)

Recently uploaded

Recently uploaded (20)

Point Cloud Processing: Estimating Normal Vectors and Curvature Indicators using Eigenvectors