The 3D Elevation Program, managed by the US Geological Survey, is acquiring quality level 2 or better light detecting and ranging (lidar) data over the conterminous United States, Hawaii, and US Territories to meet the growing need for high-resolution 3-D representations of Earth's surface, vegetation, and constructed features. OpenTopography, supported by the USGS Community for Data Integration (CDI) program, is developing well-documented and customizable Jupyter Notebook-based Python workflows for programmatically accessing, processing, and visualizing 3DEP data products for a variety of use-cases geared toward USGS applications and for users of point cloud data across the geospatial community. Presented by Cole Speed at 2022 July ESIP Meeting.

1. https://registry.opendata.aws/usgs-lidar/ Funding for this ongoiing work was provided by the Community for Data Integration (CDI) from the
project grant:“Enhancing usability of 3DEP data and web services with Jupyter notebooks
• User installs Python modules and runs Jupyter Notebook(s) on personal workstation.
• Performance and job size capabilities depend on local hardware.
• User runs Jupyter Notebook in Google Colaboratory environment on Google’s Cloud Platform.
• No local Python installs or processing required.
• Allocations of 12GB RAM and Google’s high performance CPU/GPUs
• Products directly downloaded to and accessed from the user’s Google Drive
Enhancing usability and utility of USGS 3D Elevation Program
(3DEP) lidar data and web services with Jupyter Notebooks
Cole Speed1,3
, Matthew Beckley1
, Christopher Crosby1
, Viswanath Nandigam2
!"#$%&"&'()"*+
Contact: cole.speed@beg.utexas.edu
1
UNAVCO Inc.; San Diego Supercomputer Center, UC San Diego; 3
Department of Geological Sciences
Jackson School of Geosciences, University of Texas at Austin
600 km
• 3DEP data are available as an Amazon Web Services Public Dataset in Entwine Point Tile (EPT) format for public use.
• Currently, >1800 datasets, >43 trillion lidar points, > 6 million km2
coverage with data acquistion ongoing.
• Project Goal: Develop accessible and usable workflows for accessing, processing, and visualizing 3DEP data
via customizable Python-based Jupyter Notebooks and cloud resources.
The 3D Elevation Program (3DEP)
ipyleaflet
Project Overview
• The 3D Elevation Program, managed by the US Geological Survey, is acquiring quality level 2 or better light
detecting and ranging (lidar) data over the conterminous United States, Hawaii, and US Territories to meet the
growing need for high-resolution 3-D representations of Earth's surface, vegetation, and constructed features.
• OpenTopography, supported by the USGS Community for Data Integration (CDI) program, is developing
well-documented and customizable Jupyter Notebook-based Python workflows for programmatically accessing,
processing, and visualizing 3DEP data products for a variety of use-cases geared toward USGS applications and
for users of point cloud data across the geospatial community.
Example Use-Cases of Jupyter Notebook 3DEP Workflows
z
Distribution, Installation, and Execution
Python Workflows for Accessing, Processing, and Visualizing 3DEP Data
Lidar Point Cloud
CHM
DTM
by
USGS
Watershed
(12
Digit
HUC
Service)
Cutsom
DTM/DSMs
for
USGS
7.5
x
7.5’
Quadragles
Canopy
Height
Model
for
User-defined
AOI
https://github.com/cmspeed/OT_3DEP_Workflows
Topographic
Differencing
of
3DEP
Datasets
We are seeking suggestions for specific use-cases!
USGS 3DEP
7.5 x 7.5’ Quadrangle
DTM
DTM
Differenced DSM
USGS 3DEP
12-Digit USGS WBD
USGS 3DEP
User-defined AOI
USGS 3DEP (2013)
USGS 3DEP (2011)
User-defined AOI
Lidar Point Cloud
5 km
5 km
5 km
Colorize Point Clouds by
Corresponding NAIP Imagery
NAIP Image Service
Correspoinding 3DEP Datset
Colorized point clouds
Text about querying 3DEP
Colorized point clouds
1 km USGS 3DEP
User-defined AOI
1 km
1 km
1 km
5 km
3 km
DSM
DTM
Canopy Height (m)
0 5
2 km
USGS Ground Class - Lidar Points
5 km
USGS Ground Class - Lidar Points
DTM
5 km
Get 3DEP Polygons via API
Construct PDAL Pipelines Execute the PDAL Pipelines
Define AOI on Interactive Map
API request to get up-to-date 3DEP
polygons from USGS lidar repository
(https://github.com/hobu/usgs-lidar)
Extract names, geometries, and
point counts for each dataset
AOI subset of 3DEP data is
downloaded from aws usgs-lidar
Entwine Point Tile (EPT) bucket
Processing stages are applied
Products saved locally
or to Google Drive
Workflow leverages PDAL
(Point Data Abstraction Library)
Customizable PDAL pipelines:
specify resolution and processing
steps (e.g., filtering, reclassifying,
colorizing, gridding)
iPyLeaflet interactive maps / draw
methods / shapefiles for user AOI
Estimate point density for AOI
iPywidgets promote customization
and ease-of-use
Option 2: Install and Execute on Google Colaboratory
Option 1: Local Installation and Execution
QR Code to
Github Repository
2013 - DTM
2011 - DTM Topographic Difference (2013-2011) (m)
-0.5 0.5 (m)
2 km
1 km
3DEP Point Cloud and Digital
Terrain Model (DTM)
Diamond Head Volcano, HI
Colorized 3DEP
Point cloud