Streamlining Python Development: A Guide to a Modern Project Setup
2017 ASPRS-RMR Big Data Track: The National Map
1. + 1
+
Carol Lydic
National Map Liaison for Colorado &
New Mexico
U.S.
Geological
Survey
National
Geospatial
Program
2. + 2
Slide 2
National Geospatial Program
The NGP provides a foundation of digital geospatial data
representing the topography, natural landscape, and built
environment of the US
These data and derived products and services can be accessed
through TNM Data Download and Visualization Services
Just some of the major programs:
3D Elevation Program (3DEP)
US Board on Geographic
Names (BGN) and Geographic
Names Information System
(GNIS)
National Hydrography Dataset (NHD) & Watershed Boundary Dataset (WBD)
3. + 3
+ 3
Apply lidar technology to map bare earth and 3D
data of natural and constructed features; increase
the quality level of lidar being acquired to enable
more accurate understanding, modeling, and
prediction
Goal to complete acquisition of national lidar
coverage with IfSAR in Alaska in 8 years
Address the mission-critical requirements of 34
Federal agencies, 50 states, and other
organizations documented in the National
Enhanced Elevation Assessment
ROI 5:1, conservative benefits of $690 million/year
with potential to generate $13 billion/year
Leverage the capability and capacity of private
industry mapping firms
Achieve a 25% cost efficiency gain by collecting
data in larger projects
Completely refresh national elevation data holdings
with new products and services
3D Elevation Program (3DEP)
5. + 5
+ 53D Elevation Program
Mission Critical Applications
Geologic Hazards
Aviation Safety
Annual Benefits
Rank Business Use Conservative Potential
1 Flood Risk Management $295M $502M
2 Infrastructure and Construction Management $206M $942M
3 Natural Resources Conservation $159M $335M
4 Agriculture and Precision Farming $122M $2,011M
5 Water Supply and Quality $85M $156M
6 Wildfire Management, Planning and Response $76M $159M
7
Geologic Resource Assessment and Hazard
Mitigation
$52M $1,067M
8 Forest Resources Management $44M $62M
9 River and Stream Resource Management $38M $87M
10 Aviation Navigation and Safety $35M $56M
:
20 Land Navigation and Safety $0.2M $7,125M
Total for all Business Uses (1 – 27) $1.2B $13B
Flood Risk Management
Infrastructure
NEEA Refresh is underway in partnership with NOAA
Improve documentation of requirements and benefits
based on what users know and need today
Plan for next cycle after national coverage is achieved
– what repeat rate and QLs are needed?
6. + 6
+ 6
3DEP For America’s Infrastructure
The significant challenge of improving the Nation’s
infrastructure depends on high-quality elevation data
Lidar point cloud (top) and a derived bare-earth
digital elevation model (bottom) for Denver, CO
Conservative annual benefits estimated at $170M
▪ Route, grade, line-of-sight, and utility surveys and corridor
mapping
▪ Terrain and other obstruction identification
▪ Dam, levee, and coastal-structure failure modeling and
mitigation
▪ Hydraulic and hydrologic modeling
▪ Evaluations of geologic, coastal, and other natural hazards,
and geotechnical evaluations
▪ Permit application and construction plan development and
evaluation
▪ Drainage issues and cut-and-fill estimate requirements
▪ Vegetation, topographic, and geomorphologic feature
analysis
▪ As-built model development
▪ Preliminary engineering, estimate development, and
quantity estimation activities
▪ Bridge site selection
▪ Base-map and elevation model creation
8. + 8
+ 8
3DEP Governance
■ 3DEP Executive Forum
■ Facilitates executive collaboration on
strategies to fund and implement 3DEP for
the benefit of all its stakeholders
■ Provides direction to 3DEP Working Group
■ 3DEP Working Group
■ Coordinates implementation of 3DEP
Member Agencies
Bureau of Land Management
Department of Homeland Security
Department of Transportation
Environmental Protection Agency
Federal Aviation Administration
Federal Communications Commission
Federal Emergency Management Agency
US Forest Service
US Fish and Wildlife Service
National Oceanic and Atmospheric Administration
National Park Service
Natural Resources Conservation Service
Office of Surface Mining Reclamation and
Enforcement
US Department of Agriculture
US Army Corps of Engineers
US Geological Survey
American Association of State Geologists
National States Geographic Information Council
9. + 9
+ 9
3DEP Data Acquisition
Partnership Process
Together determine
acquisition plan for
the year
Non-Federal
Broad Agency
Announcement (BAA)
Fair and equitable process
for non-Feds to partner with
Federal Agencies
Publicly announced
Competitive, clear criteria
Can include Federal
Agencies
Partners can propose to use
USGS contract (GPSC) or
their own contract
Federal
Partners =
3DEP Working
Group
Federal interagency
agreements
10. + 10
+ 10
Next Round of BAA Projects
Timeline
■ Target release date of solicitation: August 15th
■ Initial proposal due date: October 20th
■ Decisions November through January based on the availability
of funding
■ Public Webinar Dates:
■ August 10, 2017 at 1pm ET - registration required
■ August 17, 2017 at 3pm ET - registration required
see nationalmap.gov/3dep/FY18BAA.html
Or cms.geoplatform.gov/elevation/3DEP/PublicMeetings
14. + 14
+ 14
USGS Lidar Base Specification v1.2
■ All 3DEP projects are required to adhere to the specification
■ All GPSC task orders reference the specification
■ Specification covers
■ Definitions of Quality Level
■ Vertical Accuracy
■ Classification Scheme
■ Ground conditions
■ MUCH MUCH MORE!
■ Version 1.3 is in development
nationalmap.gov/3dep
Look in “Resources” on the left navigation bar
15. + 15
U.S. Interagency Elevation Inventory and
Seasketch: Find data and partners
NOAA sponsored Seasketch site
fedmap.seasketch.org
USIEI
coast.noaa.gov/inventory
16. + 16
+ 16
3DEP National Multiyear Plan
Background
3DEP Executive Forum tasked the 3DEP
Working Group to develop plan to:
Move from an annual, opportunistic process
to a unified multi-year plan
Move from patchwork irregular acquisition
footprints to a defined planning and delivery
unit
Implement a phased approach beginning in
FY18
Benefits
Facilitate greater investments and leveraging
through longer planning lead times
Defined units facilitate planning and
understanding costs, allow for improved
reporting and justification of investments
Presents a plan for nationwide coverage
Move from this…
…to something more like this
17. + 17
+ 17
3DEP National Multiyear Plan
Goals
Identify and document plans of Federal agencies to acquire 3DEP data,
distinguishing between funding plans vs. areas of interest
FY18 – collect 2-year plans from a few key agencies
Outyears – work towards planning for national coverage over multiple years
Develop a cost share model for Federal investments
Determine Federal investments on a state-by-state basis
Overall concept is 2/3 Federal and 1/3 state funding across the nation, but
ratios will vary depending on presence of Federal lands and other Federal
data needs in each state
National Plan made up of 50+ State Plans - Work with States and other non-
Federal partners to develop joint state-based plans to leverage Federal and non-
Federal funds to complete national coverage
Manage the collective plans and program information on the GeoPlatform and
Elevation Community pages
18. + 18
+ 18
3DEP Multi-Year Planning (FY18 – FY20)
PRELIMINARY Federal Plan
Based on input
from Federal
agencies
■ Assumes a level
budget
■ Identifies
projects with a
minimum/ partial
planned funding
■ Will be adjusted
annually and
based on
funding
19. + 19
+ 19
Albers Equal Area projection (EPSG:6350), XYZ units in meters
Each tile is 1 square km in area
A standard national tiling naming convention that represents the XY location for each tile
Tiles can be grouped or block nested by various attributes (counties/states/HUCs), but
each tile should be part of one and only one group (1:1 relationship)
Hawaii and territories use 1 km tile on whatever projection makes sense
Alaska continues to use 1-degree cells
Implementing in BAA GPSC projects FY18 and cooperative projects in FY19
Benefits
1 square km is small enough to approximate watersheds, county and state boundaries, etc.,
without adding a lot of area to projects
Tile sizes are equal in area no matter their location in latitude or longitude
Avoids slivers and unnecessary overlap between projects
More orderly approach to nationwide coverage
The MASTER 1k x 1k tile scheme will be provided online for public use and download
Conversion services to be developed in the future
National Tiling Scheme for Lower 49
3DEP National Multiyear Plan
20. + 20
Slide 20
Overview
One of the cornerstones of the National
Geospatial Program
Developed and maintained through
partnerships
Collaborative effort among the USGS,
Federal, State, and local partners to
improve and deliver topographic
information for the Nation
Nationwide repository of integrated data
from these sources
The National Map provides 3DEP, NHD,
WBD and other topographic information via
web visualization, services, and
downloadable data
as