This document discusses the EarthServer-2 project, which aims to create a digital Earth platform for multi-disciplinary geospatial applications. The platform includes several data services that provide access to Earth observation data through standardized interfaces like OGC services. It allows users to access, visualize, subset, combine and process vast amounts of geospatial data simultaneously from multiple data sources. The platform is demonstrated through examples of using vegetation, precipitation and soil moisture data to study drought in Eastern Africa. It also discusses lessons learned from implementing the cube technology and providing access to data through various user interfaces tailored for different user groups.

1. EARTHSERVER-2
EODATASERVICE.ORG
DIGITAL EARTH PLATFORM TO ENABLE MULTI-
DISCIPLINARY GEOSPATIAL APPLICATIONS
Simone Mantovani – Mantovani@meeo.it
MEEO – Ferrara, Italy

2. CONCLUSIONS: STEP0
THE SCOPE OF THIS PRESENTATION IS
• TO SHARE THE LESSONS LEARNT FROM AN EARTH SCIENTIST VIEWPOINT
EUDAT Conference 2018, Porto 22-25 January 2018

3. • Operational Agile Analytics on 1+ Petabyte
space/time datacubes
• Federation of services
• EO Data Service
• Marine Data Service
• Climate Data Service
• Planetary Data Service
• Project details
• Kick-Off: May 2015
• Duration: 3years
• Partners: 6 EU + 1 US + 1 AUS
EARTHSERVER-2: HTTP://EARTHSERVER.EU
EUDAT Conference 2018, Porto 22-25 January 2018

4. Eodataservice https://eodataservice.org
land cover
pollution
ocean
climate health
weather
agriculture
Exploration
Visualization
Processing
Discovery
eodataservice is a interdisciplinary / cross domain platform
High Resolution optical satellite data
Sentinel 2
Landsat
Land Products
Vegetation Indexes (from 10m, NRT)
Land Surface Temperature (NRT)
Soil Moisture (1978, present)
Land Cover type (MODIS)
Atmospheric products
Airt Temperature (NRT)
Precipitation (NOAA Hydro Estimator) NRT
Aerosol Optical Thickness @ 1Deg NRT, 10Km, 1Km
Ocean products
Sea Surface Salinity NRT
Chlorophyll data
Sea Surface Height
Sea Surface Velocity
…

5. EFFECTIVE DATA SUBSETTING
(TECHNOLOGY VIEWPOINT)
Let’s assume we want to study drought in
Eastern Africa in the last 17 years.
We want to use time series of Vegetation Index
(NDVI), Precipitation (P) and Soil Moisture (SM).
• Identification of anomalies
• Cross-fields correlation
• Identification of similar areas
Full collection
(NDVI + P + SM) > 5 TB
In real time (user viewpoint)
17 years, one point
(NDVI + P + SM) < 1MB
Vegetation
Precipitation
Soil Moisture
EUDAT Conference 2018, Porto 22-25 January 2018

6. THE ENABLING TECHNOLOGY
SINGLE PLATFORM FOR GEOSPATIAL DATA
The technology allows
managing a large set of
geospatial information
deploying a standardized
Data Access System (DAS) in
front of the data sources
It allows accessing, visualizing,
subsetting, combining,
processing, downloading all
data sources simultaneously
Only one requirement: each
dataset shall feature position
and time tags
User-provided data
data
plots
maps
Combined
fields
Socio-economic / epidemiology
EO-based data
In-situ observations
Climate models
Meteorological models
DAS
DAS
DAS
DAS
DAS
DAS
Interoperability layer
EUDAT Conference 2018, Porto 22-25 January 2018

7. Health data
Population
distribution …
DASDAS
Other geospatial data
Sentinel
1/2/3
SMOS Landsat …
DAS DASDAS
Mission-specific data
Atmosphere,
CAMS, C3S
Vegetation …
DASDAS
Thematic data
Web based GUI Jupyter Notebook GIS tools (Arcgis, qgis, …)
Standardised data
access interfaces
allow connecting a
wide range of user
interfaces
Data remain at their own
location (multiple data
centers) with the original
data format
The deployment of a DAS in
front of each data source
enables standardised data
access
DARKSIDEBRIGHTSIDEGREYSIDE
DARKSIDEBRIGHTSIDEGREYSIDE

8. @eodatacube: enablingtechnolgy
EO Data
EO data repository
DAS
DAS (Data Access System)
The DAS component provides:
- discovery service via OpenSearch Geo
and time Extensions interface
- access service via WCS interface
OpenSearch WxS
local filesystem (or NFS, S3, …)
• DAS
• OS: CentOS 7.x (minimal)
• resources: 4 core, 8GB RAM
• HDD
• 15GB system installation
• ##GB internal catalogue
• EO data repository (local filesystem or NFS)
• Services
• Apache (port 80)
HW/SW requirements
EUDAT Conference 2018, Porto 22-25 January 2018

9. •[Data collection]
•[re-projection]
•[format conversion]
•filesystem / array database
•OGC services / API (for machine)
•EO Data Service (for humans)
•Jupyter (for scientists and experts)
•Your own tools (for YOU)
@eodatacube: buildingthe ARD? or …
EUDAT Conference 2018, Porto 22-25 January 2018

10. •OGC services / API (for machine)
•EO Data Service (for humans)
•Jupyter (for scientists and experts)
•Your own tools (for YOU)
@eodatacube: … exploitingraw data?
EUDAT Conference 2018, Porto 22-25 January 2018

11. 11
ESA Landsat Datacube Australian Geoscience
Landsat Datacube
IPT-Cloud Poland
Product(s) Landsat8 Level1 GT
• 11 spectral bands
• Band Quality Assessment
• Metadata
Landsat surface reflectance
• 8 spectral bands
• Metadata
Sentinel 2 Reflectance
• 12 spectral bands
• metadata
Spatial Coverage Europe Australia Global
Temporal Coverage Nov. 2013 - today 2013/03 - 2016/10
(growing)
2015/12 - today
Data format GeoTIFF netCDF Jpeg2000
Tiling WRS2 - path/row 100km x100km 100km x100km (MGRS)
Projection Native (UTM) EPSG3577 Native (UTM)
Resolution 30 m 25 m 10, 20, 30, 60 m
@eodatacube: The RegionalData Cubes
EUDAT Conference 2018, Porto 22-25 January 2018

12. Step Performance
per day
Performance
per scene
Download 88‘ ~3’
tar xzvf 25’ ~50”
feature layers 6’ + 15’ ~11”+30”
ingestion ~20” ~0.7”
DescribeCoverage
(catalogue update)
<1” <<1”
Daily figures example
• 24-05-2016
• Number of Landsat8 scenes: 31
• Acquisition station: MTI (29) + KSI (2)
Figures onEuropeanLandsat Data Cubeand …
EUDAT Conference 2018, Porto 22-25 January 2018

13. … Figures on Sentinel2 Data Cube
Level 1C daily figures
• Products: ~ 100.000 per day
• Volume: ~ 3 TB/day
• Coverage: global
• Data format: native jp2
• Tiling: native MGRS
• Projection: native UTM
EUDAT Conference 2018, Porto 22-25 January 2018

14. CONCLUSIONS: STEP1
THE SCOPE OF THIS PRESENTATION IS
• TO SHARE THE LESSONS LEARNT FROM AN EARTH SCIENTIST VIEWPOINT
THE DARK SIDE OF THE CUBE
• LEARNING NEW TECHNOLOGY(IES)
• INSTALL, CONFIGURE AND OPERATE SERVICES (IN A DISTRIBUTED / CLOUD ENVIRONMENT)
• LEARNING NEW SERVICE(S)
EUDAT Conference 2018, Porto 22-25 January 2018

15. DATA ACCESS / DISTRIBUTION
(EODATASERVICE, EODATACUBE, INSARITALY)
EARTH OBSERVATION FOR CLIMATE-RELATED
HEALTH RISK IN AFRICA (EOCHA)
ATMOSPHERIC SCIENCES VRE
MARINE SCIENCE VRE
ILLEGAL IRRIGATION
(RE-)INSURANCE SUPPORT IN AGRICULTURE
URBAN ENVIRONMENT MONITORING
(URBMOBI)
RENEWABLE ENERGIES (WAT-ENER-CAST)
SUPPORT HERITAGE RESILIENCE AGAINST
CLIMATE EVENTS (HERACLES)
DATA ACCESS / DISTRIBUTION
(EODATASERVICE, EODATACUBE, INSARITALY)
EARTH OBSERVATION FOR CLIMATE-RELATED
HEALTH RISK IN AFRICA (EOCHA)
ATMOSPHERIC SCIENCES VRE
MARINE SCIENCE VRE
ILLEGAL IRRIGATION
(RE-)INSURANCE SUPPORT IN AGRICULTURE
URBAN ENVIRONMENT MONITORING
(URBMOBI)
RENEWABLE ENERGIES (WAT-ENER-CAST)
SUPPORT HERITAGE RESILIENCE AGAINST
CLIMATE EVENTS (HERACLES)
DATA ACCESS / DISTRIBUTION
(EODATASERVICE, EODATACUBE, INSARITALY)
EARTH OBSERVATION FOR CLIMATE-RELATED
HEALTH RISK IN AFRICA (EOCHA)
ATMOSPHERIC SCIENCES VRE
MARINE SCIENCE VRE
ILLEGAL IRRIGATION
(RE-)INSURANCE SUPPORT IN AGRICULTURE
URBAN ENVIRONMENT MONITORING
(URBMOBI)
RENEWABLE ENERGIES (WAT-ENER-CAST)
SUPPORT HERITAGE RESILIENCE AGAINST
CLIMATE EVENTS (HERACLES)
DATA ACCESS / DISTRIBUTION
(EODATASERVICE, EODATACUBE, INSARITALY)
EARTH OBSERVATION FOR CLIMATE-RELATED
HEALTH RISK IN AFRICA (EOCHA)
ATMOSPHERIC SCIENCES VRE
MARINE SCIENCE VRE
ILLEGAL IRRIGATION
(RE-)INSURANCE SUPPORT IN AGRICULTURE
URBAN ENVIRONMENT MONITORING
(URBMOBI)
RENEWABLE ENERGIES (WAT-ENER-CAST)
SUPPORT HERITAGE RESILIENCE AGAINST
CLIMATE EVENTS (HERACLES)
DATA ACCESS / DISTRIBUTION
(EODATASERVICE, EODATACUBE, INSARITALY)
EARTH OBSERVATION FOR CLIMATE-RELATED
HEALTH RISK IN AFRICA (EOCHA)
ATMOSPHERIC SCIENCES VRE
MARINE SCIENCE VRE
ILLEGAL IRRIGATION
(RE-)INSURANCE SUPPORT IN AGRICULTURE
URBAN ENVIRONMENT MONITORING
(URBMOBI)
RENEWABLE ENERGIES (WAT-ENER-CAST)
SUPPORT HERITAGE RESILIENCE AGAINST
CLIMATE EVENTS (HERACLES)
DATA ACCESS / DISTRIBUTION
(EODATASERVICE, EODATACUBE, INSARITALY)
EARTH OBSERVATION FOR CLIMATE-RELATED
HEALTH RISK IN AFRICA (EOCHA)
ATMOSPHERIC SCIENCES VRE
MARINE SCIENCE VRE
ILLEGAL IRRIGATION
(RE-)INSURANCE SUPPORT IN AGRICULTURE
URBAN ENVIRONMENT MONITORING
(URBMOBI)
RENEWABLE ENERGY (WAT-ENER-CAST)
SUPPORT HERITAGE RESILIENCE AGAINST
CLIMATE EVENTS (HERACLES)
DATA ACCESS / DISTRIBUTION
(EODATASERVICE, EODATACUBE, INSARITALY)
EARTH OBSERVATION FOR CLIMATE-RELATED
HEALTH RISK IN AFRICA (EOCHA)
ATMOSPHERIC SCIENCES VRE
MARINE SCIENCE VRE
ILLEGAL IRRIGATION
(RE-)INSURANCE SUPPORT IN AGRICULTURE
URBAN ENVIRONMENT MONITORING
(URBMOBI)
RENEWABLE ENERGIES (WAT-ENER-CAST)
SUPPORT HERITAGE RESILIENCE AGAINST
CLIMATE EVENTS (HERACLES)
Data format: GeoTIFF
Tiling: 150x150 km2 ARD h/v
Projection: Albers Equal
Area Conic
Resolution: 30 meters
Data format: GeoTIFF
Tiling: WRS2 path/row
Projection: UTM
Resolution: 30 meters
Data format: NetCDF
Tiling: 100x100km2
Projection: EPSG3577
Resolution: 25 meters
User Interface: web portal
EUDAT Conference 2018, Porto 22-25 January 2018

16. User Interface: Jupyter
•interactive data science and scientific environment
•support several programming languages (e.g. Python, R, …)
•scalable (i.e. the processing capacity of the resources made
available to a user can be extended)
EUDAT Conference 2018, Porto 22-25 January 2018

17. User Interface: Jupyter
EUDAT Conference 2018, Porto 22-25 January 2018

18.  Provide a set of e-learning examples (e.g. data discovery, data access, data
processing, data visualization, …
User Interface: Jupyter
EUDAT Conference 2018, Porto 22-25 January 2018

19. User Interface: Jupyter
Data Discovery –Access – Processing -Visualization
EUDAT Conference 2018, Porto 22-25 January 2018

20. Add your running code
User Interface: Jupyter
EUDAT Conference 2018, Porto 22-25 January 2018

21. Add your running code
User Interface: Jupyter
EUDAT Conference 2018, Porto 22-25 January 2018

22. User Interface: OGC services
http://<endpoint>/opensearch?
&service=CSW
&version=2.0.2
&request=GetRecords
&elementsetname=full
&typenames=csw:Record
&resulttype=results
&q=<ID>
&bbox=<UL,LR>
&time=<t1,t2>
for c in
(MOD_OPTDEPLOMEAN_4326_1)
return
encode ( {
red: (char)( c > 0.1 ) * 255 + (char)( c
>= 0.07 and c <= 0.1 ) * 255 ;
green: (char)( c < 0.07 and c > 0) * 255
+ (char)( c >= 0.07 and c <= 0.1 ) * 255 ;
blue: (char)( c * 0 )
}
[t(150248)]
,"png")
WCPS
OpenSearch
http://<endpoint>/wcs2?
service=WCS&Request=GetCoverage
&version=2.0.0
&CoverageID=<ID>
&SubsetX=x(x1, x2)
&SubsetY=y(y1,x2)
&SubsetT=t(t1, t2)
&format=image/png
WCS
EUDAT Conference 2018, Porto 22-25 January 2018

23. CONCLUSIONS: STEP2
THE SCOPE OF THIS PRESENTATION IS
• TO SHARE THE LESSONS LEARNT FROM AN EARTH SCIENTIST VIEWPOINT
THE DARK SIDE OF THE CUBE
• LEARNING NEW TECHNOLOGY(IES)
• INSTALL, CONFIGURE AND OPERATE SERVICES (IN A DISTRIBUTED / CLOUD ENVIRONMENT)
• LEARNING NEW SERVICE(S)
THE BRIGHT SIDE OF THE CUBE
• ONE USER INTERFACE PER USER CATEGORY
• WEB PORTAL HELPS END-USER
• JUPYTER HELPS EXPERT USER
• OGS SERVICES HELP DEVELOPER
• EXPLOITING DATA ACROSS CLOUD INFRASTRUCTURES
• FROM DATA AVAILABILITY TO DATA USABILITY!
THANK YOU
WE ARE OPEN FOR COOPERATIONS
CONTACTUS@EODATASERVICE.ORG
MANTOVANI@MEEO.IT