grass gis point cloud exploratory data analysis an open source toolkit for point cloud data processing

1. GRASS GIS Point Cloud Exploratory Data Analysis
An Open Source toolkit for point cloud data processing
September, 2016
Robert S. Dzur
rdzur@bhinc.com
1

2. grass.osgeo.org
A reliable and flexible open source analytical geospatial processing engine
2
grassmac.wikidot.com

3. Introduction
• Exploratory Analysis?
• What/Why GRASS?
• GRASS with point cloud?
• Examples
• Future?
• Take aways
3

4. Exploratory Analysis
• Point Cloud
• Vast amounts of data
• Millions / Billions of Points
• Challenges
• Visualize
• Validate
• Analyze
• Rapid
4

5. Processing Workflow Overview
5
Map Display
2D / 3D
View
Integrate
Bash / Python
Script
#!
r.out.gdal
v.out.ogr
Export
r.* - Raster
v.* - Vector
i.* - Imagery
g.* - General
db.* - Database
t.* - Time
Analyze
r.in.lidar
v,in.lidar
Import

6. GRASS GIS - Point Cloud Data Import
$ r.in.lidar -e -o --overwrite input=<required>
output=<required> method=min zrange=5000,6000 resolution=2.0 return_filter=last
class_filter=2
-e = extend computational region based on dataset
-o = override dataset project - use grass database
-- overwrite = overwrite dataset - often needed in exploratory mode
method = choose statistic (min, mean, max…etc) for raster import binning
zrange = only import points between 5000 and 6000
return_filter = filters by first, middle, last return
class_filter = filters by classification (2 = ground)
6
Format: ASPRS - LAS*

7. Computational Region
• Raster processing significant & powerful
• Vector processsing not as important
• except when deriving vectors from raster
7

8. Interstate 25, Albuquerque, NM
Mobile LiDAR Visualization
8
G
R
A
S
SIG

9. 9
I-25, Albuquerque, NM
211.8 million points
size 7.2 GB
3 miles

10. 10
grass terminal window

11. 11
r.in.lidar graphical user interface (gui)

12. Interstate 25 Intensity Image
Moblie LiDAR (0.3 ft GSD)
0 100
feet
Assessment
12

13. Interstate 25 Intensity Image
Moblie LiDAR (0.3 ft GSD)
0 100
feet
Assessment
12

14. Interstate 25 Intensity Image
Moblie LiDAR (0.3 ft GSD)
0 100
feet
Interstate 25 Intensity Image
Moblie LiDAR (0.3 ft GSD)
0 100
feet
Assessment
12

15. Interstate 25 Intensity Image
Moblie LiDAR (0.3 ft GSD)
0 100
feet
Interstate 25 Intensity Image
Moblie LiDAR (0.3 ft GSD)
0 100
feet
Assessment
12

16. Calabacillas Arroyo, Albuquerque, NM
Change Detection
13
G
R
A
S
SIG

17. LiDAR 2010
1.4 m NPS
182 MB
Photogrammetry 20145M Points 37M Points
6-inch GSD
1.27 GB
processing time: 52
seconds
14

18. 15
highlights: r.in.lidar x 2
•r.in.lidar
• dsm
• dsm
•r.mapcalc

19. 16
r.mapcalc
r.mapcalc --overwrite expression="SGM_minus_LiDAR = SGM_2014_CAB - LiDAR_2010_CAB"
2014 2010 difference- =

20. Extract change areas (raster & vector)
differences between ±1 & ±10 feet
17

21. Santa Fe, NM
Data Validation
18
G
R
A
S
SIG

22. 19
strip 101 flight line
109.4 million points
size 3.78 GB
7 minutes and 31 seconds
strip 100 flight line
102.1 million points
size 3.54 GB
Visualize / Analyze - ~37 mi. by 0.45 mi. point clouds
Santa Fe County, NM

23. 19
strip 101 flight line
109.4 million points
size 3.78 GB
7 minutes and 31 seconds
strip 100 flight line
102.1 million points
size 3.54 GB
Visualize / Analyze - ~37 mi. by 0.45 mi. point clouds
Santa Fe County, NM

24. Spatial Distribution and Regularity
• density grid =
• 2 X design ANPS =
• 2 * 0.71 = 1.42 m
• 90 percent of the cells in the grid =
• 1 point
• Using individual (single) swaths,
• only the first return points
• located within the center part
• Excluding acceptable data voids
20

25. 21
highlights: r.in.lidar x 3
•r.in.lidar
•intensity
•dsm
•count
•r.stats
•ps.map

26. 22
Strip 101 Point Per Cell Map
4.66 ft GSD
0 0.5
miles
Strip 101 Intensity Image
4.66 ft GSD
0 0.5
miles
Strip 101 Color Relief Map
Elevation in feet
0 0.5
miles
-i intensity method=min
r.in.lidar
count
return=first

27. • number of points in cell
23
count
• Strip 100 = 0.10% = zero
• Strip 101 = 0.08% = zero
r.stats -acpl in=$map4 sort=asc sep=comma output=- > output/$map21
cat output/$map21 | grep ^0, >> output/zero_count.csv

28. 24
output - visualization
ps.map --overwrite input=output/$map22 output=output/$map23
ps2pdfwr -dPDFSETTINGS=/prepress -r1200 output/$map23
Strip 100 Strip 101
Strip_100_intensity_466ft Strip_101_intensity_466ft

29. Rio Grande, NM
1960s - River Centerline
25
G
R
A
S
SIG

30. 26
LiDAR 2014
Santa Fe County
strip 101 flight line swaths
109.4 million points
size 3.78 GB
7 minutes and 55 seconds
LiDAR 2014
Santa Fe County
strip 100 flight line swaths
102.1 million points
size 3.54 GB
4 minutes and 2 seconds
Bureau of Reclamation 1962 Photos
83.2 million points
size 2.16 GB
12 hours, 52 minutes and 55 seconds
11 square miles
11 river miles
Rio Grande Bosque Farms, NM

31. highlights: r.in.lidar x 2
27
•r.in.lidar
• intensity (ortho)
• dsm
•i.segment
• obia
•r.slope.aspect
•r.roughness.vector

32. 28
1962 Digital Surface Model (DSM)
Color Shaded Relief (elevation in meters)
0 1
kilometers
1962 Photography − U.S. Bureau of Reclamation
Point Cloud Orthophoto (0.6 m GSD)
0 1
kilometers
r.in.lidar
-i intensity dsm
1960s vintage QL2 surface

33. 29
0.51962 Object Based Image Analysis (OBIA)
threshold=0.5 (i.segment) random colors
0 1
kilometers
0.31962 Object Based Image Analysis (OBIA)
threshold=0.3 (i.segment) random colors
0 1
kilometers
0.21962 Object Based Image Analysis (OBIA)
threshold=0.2 (i.segment) random colors
0 1
kilometers
0.051962 Object Based Image Analysis (OBIA)
threshold=0.05 (i.segment) random colors
0 1
kilometers
0.11962 Object Based Image Analysis (OBIA)
threshold=0.1 (i.segment) random colors
0 1
kilometers
0.021962 Object Based Image Analysis (OBIA)
threshold=0.02 (i.segment) random colors
0 1
kilometers
i.segment

34. 30
1962 Aspect Map
Point cloud derived aspect (in degrees)
0 1
kilometers
1962 Slope Map
Point cloud derived slope (in degrees)
0 1
kilometers
r.slope.aspect
slope aspect

35. 31
1962 Topographic Dispersion Map
Surface roughness Fisher’s K parameter (r.roughness.vector)
0 1
kilometers
1962 Topographic Strength Map
Surface roughness vector strength (r.roughness.vector)
0 1
kilometers
strength fisher’s k
g.extensionr.roughness.vector

36. 31
1962 Topographic Dispersion Map
Surface roughness Fisher’s K parameter (r.roughness.vector)
0 1
kilometers
1962 Topographic Strength Map
Surface roughness vector strength (r.roughness.vector)
0 1
kilometers
strength fisher’s k
g.extensionr.roughness.vector
Grohmann, C.H., Smith, M.J. & Riccomini, C., 2011.
Multiscale Analysis of Topographic Surface Roughness in
the Midland Valley, Scotland. Geoscience and Remote
Sensing, IEEE Transactions on, 49:1200-1213. http://
dx.doi.org/10.1109/TGRS.2010.2053546

37. Boca Negra Arroyo, Albuquerque, NM
Drainage Basin Study
32
G
R
A
S
SIG

38. 33
LiDAR 2014
Santa Fe County
strip 101 flight line swaths
109.4 million points
size 3.78 GB
7 minutes and 55 seconds
LiDAR 2014
Santa Fe County
strip 100 flight line swaths
102.1 million points
size 3.54 GB
4 minutes and 2 seconds
Mid-Region Council of Governments 2010 LiDAR
187.6 million points (1.4 m NPS)
size 6.38 GB
7 minutes and 4 seconds
Boca Negra Arroyo,
Albuquerque, NM

39. highlights: r.in.lidar x 1
•r.in.lidar
• dsm
•r.watershed
• accumulation
• direction
• basin
• flow path
•v.generalize
34

40. DSM
35
Elevation: MRCOG2010 (feet) 0 5
miles

41. Flow Accumulation
36
Flow Accumulation 0 5
miles

42. 37
Basin Boundary
v.generalize
input=$map8
output=$map10
method=chaiken
threshold=6 --
overwrite

43. 37
Basin Boundary
v.generalize
input=$map8
output=$map10
method=chaiken
threshold=6 --
overwrite

44. 38
Basin Boundary - 16.3 sq. miles

45. Lubbock, TX
DSM Development - Targeted Building Collection
39
G
R
A
S
SIG

46. 40
LiDAR 2014
Santa Fe County
strip 101 flight line swaths
109.4 million points
size 3.78 GB
7 minutes and 55 seconds
LiDAR 2014
Santa Fe County
strip 100 flight line swaths
102.1 million points
size 3.54 GB
4 minutes and 2 seconds
2016 Photogrammetric Semi-global Matching Point clouds
1 file: 141 million points (3-inch GSD)
size 4.82 GB
3763 total files - 502 square miles
4 hours and 18 minutes
Lubbock, TX

47. highlights: r.in.lidar x 1
•r.in.lidar
• dsm
•r.to.vect
• polygon outline
v.hull
• generalize
v.buffer
• interior clip
•r.out.gdal
• output GeoTiff
41

48. 2016 Lubbock Digital Surface Model
Semi−global matching photogrammetric point clouds (3763)
0 10
miles
Point Cloud DSM
• Decimated Point
Cloud
• 141 M -> 4 M (2%)
• 4.6 GB -> 140 MB
• r.in.lidar
• 3.0 ft GSD
• 4 hours processing
• 6 core machine
• 10 simultaneous
GRASS databases
42

49. 43
2016 Lubbock DSM
Color Relief Map (elevation in feet)
0 0.5
miles
2015 Lubbock DSM
Color Relief Map (elevation in feet)
0 0.5
miles
r.in.lidar
2015 leaf-off 2016 leaf-on

50. 2016 Lubbock Normalized Difference Vegetation Index (NDVI)
DSM / NDVI Candidate Building Detection
Buildings
Candidate Building 0 0.25
44
New Candidate Buildings
2016 NDVI

51. • GRASS Development Team, 2015. Geographic Resources
Analysis Support System (GRASS) Software, Version 7.0. Open
Source Geospatial Foundation. http://grass.osgeo.org
• Michael Barton, PhD. - GRASS Macintosh Binaries http://
grassmac.wikidot.com/
Acknowledgements
45