DSD-INT 2015 - from foreshore data to foreshore information - Edward P. Morris

11/24/2015 From data to information
http://edwardpmorris.github.io/dsd_fast_teaching/#1 1/60
The capacity of light to carry and convey
information is perhaps its most important, and
remarkable, characteristic.
“
”
Ian Walmsley in 'Light : A very short introduction', Oxford University Press.

3/61

4/61
https://youtu.be/gF9UlTYR5d0

5/61
https://youtu.be/28hImDyJXu0?list=PLbyvawxScNbs9ET50PFwZTworudkNsnH_

Remote sensing 101
6/61

Satellite Platform
Landsat 8, Sentinel 1A, Sentinel 2A
Sensor or Instrument
OLI + TIRS, C-SAR, MSI
Resolution
Platform, sensor and resolution
Spatial: Pixel size, extent
Temporal: Revisit time
Spectral: Band width, Number of bands,
Sensitivity
·
·
·
7/61

Has both wave and particle properties at the
same time (wave-particle duality).
"Electromagneticwave3D" by Lookang many thanks to Fu-Kwun Hwang and
author of Easy Java Simulation = Francisco Esquembre - Own work. Licensed
under CC BY-SA 3.0 via Commons.
Electromagnetic radiation
Waves can be described by their
amplitude and wavelength
Polarisation: waves oscillate in more than
one orientation
Photons are the elementary particles of
eletromagnetic radiation
Photons can be interact with matter by
scattering and absorption
Reflectance is scattering in a defined
direction
·
·
·
·
·
8/61

Electromagnetic wave spectrum
"EM spectrum". Licensed under CC BY-SA 3.0 via Commons.
9/61

Atmospheric electromagnetic opacity
Gamma rays, Xrays and ultraviolet
light blocked by the upper atmosphere
(best observed from space).
Visible light
observable
from Earth,
with some
atmospheric
distortion.
Most of the
infrared spectrum
absorbed by
atmospheric
gasses (best
observed
from space).
Radio waves observable
from Earth.
Longwavelength
radio waves
blocked.
0 %
50 %
100 %
0.1 nm 1 nm 10 nm 100 nm 1 µm 10 µm 100 µm 1 mm 1 cm 10 cm 1 m 10 m 100 m 1 km
Wavelength
Atmospheric
opacity
"Atmospheric electromagnetic opacity" by NASA (original); SVG by Mysid. - Vectorized by User:Mysid in Inkscape, original NASA image from File:Atmospheric electromagnetic transmittance or
opacity.jpg.. Licensed under Public Domain via Commons.
10/61

Ocean Optics Web Book • All contents 2015 Creative Commons Attribution license.
Measures (sun) light
Passive visible and infrared
day-time only*
radiance (light) passes through
atmosphere
·
·
affected by scattering and absorption
affected by clouds
requires atmospheric correction
procedures
-
-
-
11/61

The surfaces of substances have specific
scattering and absorption properties
Red and green light is absorbed,
blue reflected.
"Simple reflectance" by User:Phidauex - Own work. Licensed under Public Domain via
Commons.
allows identification of surfaces (i.e, the
blue surface)
allows quantification of substances (i.e.,
the number of blue 'rocks')
·
·
12/61

Water strongly absorbs light giving it a distinctive reflectance spectrum that can be
summarised by spectral indices (ratios), e.g., the 'Normalised Difference Water Index'
(NDWI, McFeeters et al. 1996).
"Absorption spectrum of liquid water" by Kebes at English Wikipedia. Licensed under CC BY-SA 3.0 via Commons.
13/61
NDWI = (NIR - Green)/(NIR + Green)

All photosynthetic organisms contain chlorophyll giving them a distinctive reflectance
spectrum that can be summarised by spectral indices (ratios), e.g., the 'Normalised
Difference Vegetation Index' (NDVI).
"Vegetation spectral response" by wiki.landscapetoolbox.org. Licensed under Public Domain via Commons.
14/61
NDVI = (NIR - Red)/(NIR + Red)

The different pigments of photosynthetic organisms result in different 'colours',
potentially allowing identification of foreshore vegetation types.
wikipedia.org/wiki/Chloroplast.
15/61

Colorvision Cadiz Bay
16/61

"Fig. 6 Relationship between NDVI and chlorophyll..." by Kromkamp et al.
2006.
"Time-ensemble (2000-2015) mean modified normalised difference water
index (MNDWI) of Cadiz Bay derived from the Landsat (USGS) archive" by
FAST project.
Spectral ratios, such as NDWI and NDVI can be used to quantify properties of
foreshores.
17/61

Synthetic Aperture Radar (SAR)
"A geometric model of a SAR system" by Zhou et al. 2009.
Active radio
active radio pulses sent and collected
effective antena size much larger using
movement of satellite and clever
processing (Synthetic Aperture).
measures polarised radio waves e.g.,
horizontal and vertical.
allows increased spatial resolution.
·
·
·
·
18/61

Active Radio
Synthetic Aperture Radar (SAR)
"Commonly used frequency bands..." by Moreira et al. 2013.
most common bands are L, C and X
different band frequencies are suited to different applications
·
·
penetrating canopies: L and P, very high resolution: X, multi-purpose: C-
19/61

"Surface and volume scattering of a SAR beam for trees " by (Fernandez-
Ordonez et al. 2009.).
Surface interactions
Active Radio
still water tends to be a specular reflector
of radio waves i.e, usually low backscatter.
complex interactions with vegetation
result in different return signals.
buildings often have strong backscatter
signal.
interactions vary with beam polarization
i.e., different polarizations provide more
information.
·
·
·
·
20/61

"Speckle occurs in SAR images due to the coherent sum of many elemental
scatterers within a resolution cell..." by Moreira et al. 2013.
Speckle
Active Radio
caused by the many scatterers in a 'pixel'.
coherence of the scattered signal results
in strong fluctuations across the image.
'multi-looking' used to reduce speckle at
cost of spatial resolution.
adaptive filtering can also be used to
reduce speckle.
·
·
·
non-coherent averaging of the
intensity image.
-
·
21/61

Active Radio
"Interferometric Wide Swath Dual-Polarisation Sentinel-1 acquired on 2nd December 2014, band combination: VH-VV-VH (RGB) over the Sundarbans Mangrove, India and Bangladesh
(2014 / 2015) / ESA Image © Satellite Application Catapult Ltd, 2014.
22/61

Further resources
General:
Passive visible and infrared:
Active radar (SAR):
Glossary of remote sensing terms
EO portal database of satellite missions
Landscape Toolbox Wiki
·
·
·
Ocean optics web book·
Moreira, A., Prats-Iraola, P., Younis, M., Krieger, G., Hajnsek, I., & Papathanassiou, K. P.
(2013). A tutorial on synthetic aperture radar. Geoscience and Remote Sensing Magazine,
IEEE, 1(1), 6-43.
SAREDU; full open-access course on SAR by DLR
·
·
23/61

How to access remote sensing data
24/61

Data availability
Some earth observation satellites with open access data, based on wiki.landscapetoolbox.org. *C=Color, E=Elevation, H=Hyperspectral,
M=Multispectral, P=Panchromatic, R=Syntheric Aperature Radar
NAME ABBREVIATION RESOLUTION
(M)
AVAILABILITY RETURN
INTERVAL
TYPE* PLATFORM
Landsat Thematic Mapper 5 L5-TM 30,60 1984 to 2013 16 days M Satellite
Landsat Enhanced Thematic Mapper 7 L7-ETM 15,30,60 1995 to 2003 16 days P,M Satellite
Moderate-resolution Imaging
Spectroradiometer
MODIS 250,500,1000 2000 to
present
Daily M Satellite
Medium Resolution Imaging Spectrometer ENVISAT-MERIS 300 2002 to
present
3 days M Satellite
Landsat Data Continuity Mission (Landsat 8) L8-OLI 15,30,100 2013 to
present
16 days P,M Satellite
Copernicus Sentinel-1A S-1A 4,10,25,40 2015 to
present
12 (6) days R Satellite
Copernicus Sentinel-2A S-2A 10,20,60 2015 to
present
12 (6) days M Satellite
25/61

Data type and format
Landsat Missions Timeline
http://landsat.usgs.gov//about_mission_history.php.
26/61

Generally surface reflectance products are
prefered, however see L8-SR user notes and
Landsat Higher Level Science Data Products:
http://landsat.usgs.gov//landsat_level_1_standard_data_products.php.
Landsat product types
not completely validated
not all scenes can be processed: Landsat 8
Pre-WRS-2 scenes (before April 11, 2013),
scenes with a solar zenith angle greater
than 76°.
Efficancy of L8SR correction will be likely
reduced in: Hyper arid or snow covered
regions, Low sun angle conditions, Coastal
regions where land area is small relative
to adjacent water, Areas with extensive
cloud contamination.
·
·
·
High latitudes (> 65º) may not be valid.·
27/61

Data format and type
Landsat Standard L1 products
Standard Terrain Correction (Level 1T -precision and terrain correction) if possible.
Cubic Convolution (CC) resampling method.
30-meter (TM, ETM+, OLI) and 60-meter (MSS) pixel size (reflective bands).
Universal Transverse Mercator (UTM) map projection (Polar Stereographic projection for
scenes with a center latitude greater than or equal to -63.0 degrees).
World Geodetic System (WGS) 84 datum.
MAP (North-up) image orientation.
GeoTIFF per band, quality assement + metadata (.MTL)
·
·
·
·
·
·
·
28/61

Data format and type
Landsat Higher Level Science Data products
surface reflectance
surface reflectance spectral indices
transport format as standard GeoTIFF or ENVI binary or HDF-EOS2, see guide
·
includes basic classification (water, cloud) and per pixel quality mask
metadata is .xml (schema: http://espa.cr.usgs.gov/v1.2)
-
-
·
·
29/61

Sentinels Timeline
ESA, http://www.unoosa.org/pdf/pres/stsc2015/tech-53E.pdf. ESA, http://www.unoosa.org/pdf/pres/stsc2015/tech-53E.pdf.
30/61

Sentinel 1 product types
https://sentinel.esa.int/web/sentinel/missions/sentinel-1/data-products.
31/61

https://earth.esa.int/web/sentinel/user-guides/sentinel-1-sar/data-
formats/sar-formats.
The SENTINEL-SAFE format wraps a folder
containing image data in a binary data format
and product metadata in XML.
Sentinel 1 product format
32/61

Sentinel 2 product types
https://sentinel.esa.int/web/sentinel/missions/sentinel-2/data-products.
33/61

https://sentinel.esa.int/web/sentinel/user-guides/sentinel-2-msi/data-
formats.
The SENTINEL-SAFE format wraps a folder
containing image data in a binary data format
and product metadata in XML.
Sentinel 2 product format
34/61

http://earthexplorer.usgs.gov/.
Earth Explorer
official bulk dissemination system for the
Landsat (and other) products.
requires registration (quick and easy);
email address used to send download
links!
no restrictions on data use but requested
to acknowledge source:
·
·
·
USGS Products: 'Data available from
the U.S. Geological Survey.'
NASA LP DAAC Products: 'These data
are distributed by the Land Processes
Distributed Active Archive Center (LP
DAAC), located at USGS/EROS, Sioux
Falls, SD. http://lpdaac.usgs.gov'
-
-
surface reflectance products generated
'on demand'.
·
35/61

https://scihub.esa.int.
Sentinel Scientific Data Hub
official bulk dissemination system for the
Sentinels products
requires registration (quick and easy)
use of data implies acceptance of Terms
and Conditions of the use and distribution
of Sentinel data
no limitations but must attribute
'Copernicus Sentinel data [Year]' or
'Contains modified Copernicus Sentinel
data [Year]'
·
·
·
·
36/61

Other examples of how to get the data
General:
Sentinels:
Google Earth Engine (GEE); Landsat archive, Sentinel-1 (available late 2015?), MODIS, MERRIS,
ect. Sentinel-2 (planned).
·
Google Storage also mirrors much of these archives. You can install their “gsutil” (free)
and then list this directory, which is indexed by sensor (for example, L8 is Landsat 8) and
path/row: gsutil ls gs://earthengine‐public/landsat/
Note Sentinel-1 has undergone further pre-processing in GEE
-
-
Sentinel Data Access Service (SEDAS); useful portal by UK Space Agency·
37/61

Other examples of how to get the data
Landsat:
Via commandline using curl:
USGS Earth Resources Observation and Science (EROS) Center Science Processing
Architecture (ESPA) On Demand Interface for higher level products and processing options
Landsat Look for quick looks and standard data
GloVis an alternative to EarthExplorer
Python Landsat-Download
Python Landsat-util
·
·
·
·
·
1. # download a specific sentinel product
2. S:>curl ‐u User:Password ‐JO "https://scihub.esa.int/dhus/odata/v1/
3. Products('b94d872a‐ef4f‐4117‐8a6b‐1a90466ae785')/$value"
38/61

How to extract information
39/61

Basic concepts
Georectification: Mapping the data to real world coordinates.
Calibration: Converting the data to real world units
Morphological operations: Filtering, changing resolution, ect.
Course classification: Assigning different classes (e.g., clouds, shadows, water) using
segmentation or clustering.
at-instrument; instrument-specific digital numbers to units , i.e., radiance
top-of-atmosphere; accounting for platform to target geometry.
surface; removing atmospheric affects and corrections for topography
·
·
·
40/61

with GUI: Packages and frameworks:
Open source image processing (some options)
Sentinel Application Platform (SNAP);
official software for processing sentinel
data, integrates NEST and BEAM.
Quantum GIS (QGIS); extensible GIS
system with many plugins linking to image
processing packages (BEAM plugin).
BEAM
SeaDas
Monteverdi2
GRASS
·
·
·
·
·
·
Google Earth Engine; JavaScript and
Python
Orpheo Toolbox (OTB); C
RSGISLib; Python
OpenCV; C, Python
raster; R
·
·
·
·
·
41/61

Sentinel Application Platform (SNAP)
A common architecture for all Sentinel Toolboxes
open-source (Java) available at GitHub or precompiled at STEP.
successor of the mature BEAM + NEST software.
efficient parralization of calculations resulting in good performance on modern PCs.
Python bindings available soon?
Sensor specific toolboxes (Sentinels, but also many other sensors).
Developed by Array Systems and DLR (S-1), C-S (S-2) and Brockmann Consult (S-3) for ESA.
·
·
·
·
·
·
42/61

Sentinel Application Platform (SNAP)
http://step.esa.int/main/toolboxes/snap/.
43/61

Getting help
Detailed in built help
Toolbox tutorials, also consider NEST and BEAM tutorials.
SNAP forum; a shared user experience
SNAP development; how to contribute
·
·
·
·
44/61

The power of the Graph Builder and Graph
Processing Tool (GPT)
graphical building of workflows [graphs]
save workflows in XML and share
apply to single image or batch process many images
run from within the GUI or the command line (GPT)
·
·
·
·
45/61

ESA Research and Service Support (RSS)
'RSS has the mission to support the Earth Observation (EO) comunity in exploiting EO data'
supports institutions, scientists and developers.
supports data provisioning, access and processing.
help to scale up workflows
computation on demand and cloud computing service free for 'non-profit' organisations.
·
·
·
·
contact rss_team@esa.int to check conditions and egibility.-
46/61

Acknowledgents
The research leading to these results has received funding from the
European Union Seventh Framework Programme (FP7/2007-2013)
under grant agreement n° 607131. All views presented are those of the
author’s, the European Union is not liable for any use that may be
made of the information contained therein.
The FAST project team for support and contributions.
Landsat data courtesy of USGS.
This presentation includes Copernicus Sentinel data 2015.
Videos from NASA and ESA YouTube channels.
Comic design and production; The Pilot Dog
Slide deck made using slidify
·
·
·
·
·
·
47/61

Get this talk:
Find out more about FAST
www.fast-space-project.eu
@FP7FAST
Facebook FastSpaceProject
·
·
·
slides:
http://edwardpmorris.github.io/dsd_fast_teaching
source:
https://github.com/edwardpmorris/dsd_fast_teaching
·
·
48/61

Now your turn...
49/61

Using Sentinel 1 toolbox (S1TBX) to get information
about coastal systems
Introduction to the Sentinel 1 toolbox (S1TBX)
This workshop will cover the basics of working with S1TBX to process Sentinel-1 data:
open S1A-IW-GRDH product
basic pre-processing
export data
·
·
·
50/61

Using the Graph Builder in S1TBX
use the graph builder to save your workflow.
take a look at the graph XML format.
edit the XML so as apply different types of speckle filter and multilooking, compare results.
set up thresholding of the image using band-math and extract 'water'.
run a basic classification and compare to the threshold result.
·
·
·
·
·
51/61

Open S1A-IW-GRDH product
File > Open Product OR Import Raster Data > SAR data > SENTINEL‐1
52/61

Examine the S1A-IW-GRDH product
Products View expand structure
Open window to display raw data (RIGHT CLICK BAND > Open Image Window)
53/61

Subset S1A-IW-GRDH product
Zoom in on area of interest; note image is mirrored
Raster > Subset OR RIGHT CLICK ON IMAGE > Spatial Subset From View
54/61

Apply at-sensor and top-of-atmosphere calibration
SAR Processing > Radiometric > Calibrate
55/61

Apply a 'speckle' filter
SAR Processing > Speckle Filtering
56/61

Apply a geometric correction
SAR Processing > Geometric > Elipsoid Correction
57/61

Convert to decibels
Utilities > Dataset Conversion > Linear to/from dB
Raster > Dataset Conversion > Linear to/from dB
58/61

Examine the pre-processed product
Look at a profile from water to land:
Analysis > Profile Plot
Add a vector layer:
Vector > Import OR Line Drawing Tool
Look at a histogram for the scene:
Analysis > Histogram
59/61

Export the pre-processed product
Choose your prefered export format
File > Export Raster Data > NetCDF4‐CF
60/61

Using the Graph Builder in S1TBX
Tools > Graph Builder
61/61

DSD-INT 2015 - from foreshore data to foreshore information - Edward P. Morris

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to DSD-INT 2015 - from foreshore data to foreshore information - Edward P. Morris

Similar to DSD-INT 2015 - from foreshore data to foreshore information - Edward P. Morris (20)

More from Deltares

More from Deltares (20)

Recently uploaded

Recently uploaded (20)

DSD-INT 2015 - from foreshore data to foreshore information - Edward P. Morris