Presentation to ESA Big Data From Space (BiDS2014), November 2014. Big data from space requires processing large amounts of data in a distributed environment. For efficient, quality and cost-effective deployment, these environments must be based on open standards. The Open Geospatial Consortium (OGC) open standards for geospatial-temporal information have been tuned through implementations to meet the needs of big data.

1. ®
OGC
Big Data and
Open Geospatial-Temporal
Information Standards
George Percivall
OGC Chief Engineer
2014 Conference on Big Data from Space (BiDS '14)
Jointly organised by ESA, SatCen, JRC
12-14 November 2014, ESRIN, Frascati, Italy
© 2014 Open Geospatial Consortium

2. OGC ®
OGC Services Architecture
• Introduction to OGC Standards
• …Service Interfaces and Encodings standards
• …Grouped into semantically similar sets

3. OGC ®
OGC Services Architecture
WMS
WFS WMTS
Simple
Features
Access
Access Services
Geospatial
Feature Data
Geospatial
Browse/Maps
WCS
Geospatial
Coverage Data

4. OGC ®
OGC Services Architecture
WMS
WFS WMTS
Simple
Features
Access
Access Services
Geospatial
Feature Data
Geospatial
Browse/Maps
WCS
Geospatial
Coverage Data
SPS SensorML O&M SOS
Sensors
Puck
Sensor Web Enablement
Discover Task Access

5. OGC ®
OGC Services Architecture
Processing Services
TJS WPS WCPS OpenMI
Other
Data
WMS
WFS WMTS
Simple
Features
Access
Access Services
Geospatial
Feature Data
Geospatial
Browse/Maps
WCS
Geospatial
Coverage Data
SPS SensorML O&M SOS
Sensors
Puck
Sensor Web Enablement
Discover Task Access

6. OGC ®
OGC Services Architecture
Processing Services
TJS WPS WCPS OpenMI
Other
Data
Discovery Services
Geospatially
Enabled
Metadata
CSW OpenSearch
Geo ebRIM
WMS
WFS WMTS
Simple
Features
Access
Access Services
Geospatial
Feature Data
Geospatial
Browse/Maps
WCS
Geospatial
Coverage Data
SPS SensorML O&M SOS
Sensors
Puck
Sensor Web Enablement
Discover Task Access

7. Other Services
Workflow, Alerts
OGC ®
OGC Services Architecture
Processing Services
TJS WPS WCPS OpenMI
Other
Data
Discovery Services
Geospatially
Enabled
Metadata
CSW OpenSearch
Geo ebRIM
WMS
WFS WMTS
Simple
Features
Access
Access Services
Geospatial
Feature Data
Geospatial
Browse/Maps
WCS
Geospatial
Coverage Data
SPS SensorML O&M SOS
Sensors
Puck
Sensor Web Enablement
Discover Task Access

8. OGC Services Architecture
NetCDF GMLJP2
IndoorGML GeoSparql
Other Services
Workflow, Alerts
OGC ®
SLD
WaterML SE
GeoPackage
Data Models
WMC and Encodings
FE
GML
GeoXACML
KML
CityGML OpenGeoSMS
Processing Services
TJS WPS WCPS OpenMI
Other
Data
Discovery Services
Geospatially
Enabled
Metadata
CSW OpenSearch
Geo ebRIM
WMS
WFS WMTS
Simple
Features
Access
Access Services
Geospatial
Feature Data
Geospatial
Browse/Maps
WCS
Geospatial
Coverage Data
SPS SensorML O&M SOS
Sensors
Puck
Sensor Web Enablement
Discover Task Access

9. OGC Services Architecture
Visualization / Decision Tools and Applications
NetCDF GMLJP2
IndoorGML GeoSparql
Other Services
Workflow, Alerts
OGC ®
GeoAPI
OpenLS
SLD
WaterML SE
GeoPackage
Data Models
WMC and Encodings
FE
GML
GeoXACML
KML
CityGML OpenGeoSMS
Processing Services
TJS WPS WCPS OpenMI
Other
Data
Discovery Services
Geospatially
Enabled
Metadata
CSW OpenSearch
Geo ebRIM
WMS
WFS WMTS
Simple
Features
Access
Access Services
Geospatial
Feature Data
Geospatial
Browse/Maps
WCS
Geospatial
Coverage Data
SPS SensorML O&M SOS
Sensors
Puck
Sensor Web Enablement
Discover Task Access

10. ESA HMA and OGC
• Foreword by Mark Reichardt,
OGC CEO and President
– Comprehensive guide to the use of
international standards to enable a
European EO system of systems.
– “Indispensable resource for organisations
with EO assets that wish to connect to and
build a local to global system of systems”
• OGC Working Groups of major importance
to HMA
– Service Architecture
– Sensor Web Enablement
– Catalogue
– Metadata
– Coverages
– Compliance
OGC ®
Thomas Usländer (Fraunhofer IOSB)
Yves Coene (Spacebel s.a.)
Pier Giorgio Marchetti (ESA)
© 2014 Open Geospatial Consortium 10

11. ESA HMA Protocols under Configuration Control
http://wiki.services.eoportal.org/tiki-index.php?page=HMA+Configuration+Management+Table
Service Standard Status
Collection Metadata OGC 11-035r1, EO Collection and Service Discovery using the ebRIM Application Profile of CSW 2.0, version
OGC ®
© 2014 Open Geospatial Consortium 11
1.0, 26/03/2013.
OGC Best Practice
Collection& Service
Discovery
OGC 13-084, I15 (ISO19115 Metadata) Extension Package of CSW ebRIM profile, version 1.0, 28/04/2014 OGC Implementation
Specification
OGC 10-032r8, OpenSearch Geo and Time Extensions, version 1.0.0, 14/04/2014. OGC Implementation Standard
OGC 13-026r5, OpenSearch Extension for Earth Observation, DRAFT 1.0.0, 29/10/2014 OGC SWG Internal document
Earth Observation
Product Metadata
OGC 06-080r4, GML Application Schema for EO Products, Version 1.0.0, 25/02/2010
OGC Profile
OGC 10-157r3, Earth Observation Metadata Profile of Observations and Measurements,Version 1.0, 12/06/2012 OGC Implementation Standard
Product Discovery OGC 06-131r6, EO Products Extension Package for ebRIM Profile of CSW 2.0, Version 1.0.0, 10/02/2010 OGC Implementation Standard
OGC 10-032r8, OpenSearch Geo and Time Extensions, version 1.0.0, 14/04/2014. OGC Implementation Standard
OGC 13-026r5, OpenSearch Extension for Earth Observation, DRAFT 1.0.0, 29/10/2014 OGC SWG Internal document
Order OGC 06-141r6. Ordering Services Framework for Earth Observation Products Interface Standard, Version 1.0.0,
09/01/2012
OGC Implementation
Specification
OGC 13-042, RESTful Encoding of Ordering Services Framework for Earth Observation Products, version 1.0,
28/04/2014.
OGC Best Practice
Feasibility Analysis OGC 10-135, Sensor Planning Service Interface Standard 2.0 Earth Observation Satellite Tasking Extension,
version 2.0, 28/03/2011
OGC Implementation Standard
OGC 13-039, OpenSearch Extension for Earth Observation Satellite Tasking, OGC Best Practice
OGC 14-012r1, RESTful encoding of Sensor Planning Service for Earth Observation Satellite Tasking, version
OGC Best Practice
1.0.0, 17/07/2014.
Web Map Service
(WMS)
OGC 07-063r1 OpenGIS WMS EO Application Profile Version 0.3.3, 05/11/2009 OGC Best Practice
Web Coverage
Service (WCS)
OGC 09-110r4 WCS 2.0 Interface Standard - Core, Version 1.0.0, 10/07/2012.
OGC 09-146r2 GML 3.2.1 Application Schema - Coverages (1.0.1)
OGC 09-147r3 WCS 2.0 Interface Standard - KVP Protocol Binding Extension (1.0.1)
OGC 09-149r1 WCS 2.0 Interface Standard - XML/SOAP Protocol Binding Extension (1.0)
OGC 09-148r1 WCS 2.0 Interface Standard - XML/POST Protocol Binding Extension (1.0)
OGC 09-153 WCS 2.0 Overview: Core and Extensions (Best Practice)
OGC 10-164, OGC GML 3.2.1 AS – Coverages – CF NetCDF Coverage Encoding Extension, version 2.0.0
OGC 12-108, OGC GML 3.2.1 AS – Coverages – GML JPEG2000 Coverage Encoding Extension, version 0.0.4.
OGC Interface Standard
OGC 12-100r1 OGC GML 3.2.1 Application Schema – Coverages – GeoTIFF Coverage Encoding Profile, v.1.0 OGC Interface Standard
OGC 10-140r1 OGC Web Coverage Service 2.0 Interface Standard - Earth Observation Application Profile
OGC Interface Standard
v.1.0.0, 26/02/2014.
Web Processing
Service (WPS)
No standards under configuration control
Download Service OGC 13-043, Download service for EO Products, Version 1.0, 31/01/2014. OGC Best Practice
Identity
OGC 07-118r9, User Management Interfaces for Earth Observation Services, Version 1.1, 28/04/2014. OGC Best Practice
Management
!

12. OGC ®
OGC Big Data Working Group
• Open forum on how to use OGC standards for Big Data
interoperability, access, and especially analytics
– Encourage collaborative development among participants
representing many organizations and communities,
– Ensure appropriate liaisons to other Big Data relevant working
groups, both inside and outside OGC.
• Big Data agenda – September 2014
– GeoWave Introduction and Overview (NGA)
– Well Known Text for Accumulo (NGA)
– Spatio-Temporal Data Cube Analytics (Jacobs Univ.)
– Elasticity for WMS Hosted in a Cloud (Amazon)
– NOAA Big Data Partnership RFI (NOAA)
http://www.opengeospatial.org/projects/groups/bigdatadwg
© 2014 Open Geospatial Consortium 12

13. Discrete Global Grid Systems
OGC ®
Slide Source: Matt Purss, Geoscience Australia
ANZLIC
National
Nested
Grid***
SCENZ-Grid
by CSRIO for NZ
Earth System Spatial Grid
by China
Snyder Grid
e.g. PYXIS
*** See BIDS 2014 paper: Iterating Petabyte-scale Earth Observation processes in the
Australian Geoscience Data Cube, Simon Oliver (Geoscience Australia)

14. Discrete
Global
Grid
System
(DGGS)
Standards
Working
Group
(SWG)
• Develop common criteria that will
define conformant DGGSs
– Considering Goodchild criteria
• Develop conceptual standard to
facilitate data fusion between
DGGSs using OGC Standards
– to make them interoperable – with
conventional and other DGGS data
– to standardize operations on them
• Engage stakeholders to encourage
new use cases and adoption of
interoperability through DGGSs
http://www.opengeospatial.org/projects/groups/dggsswg
OGC ®

15. OGC ®
Big data in OGC Testbeds
• OGC Interoperability Program
– Prototyping unites users and industry in accelerating interoperability
– 85 initiatives since 1999: Testbeds, Pilots, Experiments
• OGC Testbeds with Big Data from space
– OWS-8: Observation Fusion
– OWS-9: OWS Innovations
– OGC Testbed 10: Open Mobility and Cloud
– OGC Testbed 11: Urban Climate Resilience
• Highlights discussed in the following categories
– Maps; Coverages; Performance
© 2014 Open Geospatial Consortium 15

16. OGC ®
Maps and Big Data
• Web Map Tiling Service (WMTS)
– WMTS is WMS tuned for high velocity maps
– Serves map tiles with predefined content, extent, and resolution
– TileMatrixSet: collection of tile matrices defined at different scales
• OWS-9 evaluated diverse raster map tiling schemes
– WMTS, Tile Map Service (OSGeo), MBTiles, TileCache
(MetaCarta), and various others
• Recommendations forwarded to OGC Standards Program.
– Basis for “WMTS – Simple” based on limited TileMatrixSet
– Revised WTMS standard nearing completion in SWG
OWS-9 Map Tiling Methods Harmonization (Editor Joan Maso, OGC Doc 12-157)
© 2014 Open Geospatial Consortium 16

17. WMTS Example: Several sources, Same TileMatrix
a.tile.openstreetmap.org mt1.google.com otile1.mqcdn.com/tiles/1.0.0/map
www.ogc.uab.cat/SITiled/GeoEye_Haiti_M
OGC ®
Copyright © 2013 Open Geospatial Consortium
tile.opencyclemap.org/cycle
ecn.t0.tiles.virtualearth.net
X=9798
Y=14664
X=9799
Y=14664
X=9800
Y=14664
X=9798
Y=14665
X=9799
Y=14665
X=9800
Y=14665

18. OGC ®
OWS-8 Coverages as Big Data
• WCS 2.0 Earth Observation Application Profile (EO-AP)
– Build on WCS 1.0 EO-AP developed in ESA’s HMA project.
• …to include data models for:
– 2-D Coverage with latitude and longitude axes, e.g., a hyperspectral satellite scene.
– Subset of a 2-D satellite scene (either radar or multispectral)
– Time series of co-registered 2-D satellite scenes
– Stitched Mosaic as a 2-D horizontal for several co-referenced non-overlapping datasets
– Dataset Series as a collection of coverages;
• …supporting these data products:
– MODIS
– ENVISAT MERIS L3 demonstration products (http://earth.eo.esa.int/level3/)
– ENVISAT ASAR Wide Swath
– Limb-scanning instruments (e.g. Microwave Limb Sounder) typically produce vertical profiles,
– Narrow-swath instruments (e.g., CALIPSO) produce vertical cross-sections.
• …and these encodings:
– GeoTIFF, netCDF / CF-netCDF, JPEG2000, HDF-EOS
• With a WCS 2.0 EO-AP Compliance Test and Reference Implementation
OWS-8 WCS 2.0 EO Application Profile ER (11-096)
OWS-8 WCS 2.0 EO Application Profile Compliance Tests and Reference Implementation ER (11-095)
OWS-8 ER Metadata Mapping between NASA ECS/HDF-EOS and WCS 2.0 (11-090)
Copyright © 2011 Open Geospatial Consortium

19. OGC ®
OWS-8 Coverage Processing
• WPS for EO data analysis
of multiple coverages and
other data
• WCPS for advanced
access to EO Coverages
– multi-coverage fusion
capabilities
– queries taken from real-life
examples
WCPS and WPS Orchestration
OWS-8 Geoprocessing of EO (Editor P. Baumann, OGC 11-116)
© 2014 Open Geospatial Consortium 19

20. OWS-8 Big Data Analytics on Motion Imagery
OGC WPS
for generating
track features
STANAG 4676
represented as
GML features
VMTI Implemented
in OWS-8
OGC ®
Display to Common Operational Picture
© 2011 Open Geospatial Consortium, Inc.
…and send to Mobile Client
References:
Ingo Simonis, IGSi OGC Doc 11-108
R. Cass, M. Simms, Compusult, OGC Doc 11-134

21. OGC ®
VMTI to GeoSMS
OWS-8 Mobile Client Implementation
• Android app for OGC GeoSMS, WFS features and KML
• Mobile client receives GeoSMS with URL for WFS track feature
Videos of OWS-8 Moving Object Tracking
http://www.opengeospatial.org/pub/www/ows8/tracking.html
Copyright © 2012, Open Geospatial Consortium

22. OWS-9 Access to Big Data Coverages
• OWS-9 WCS and OPeNDAP
– DAP and WCS compliment each other.
– Easy to use DAP to ‘reach into’ files and access their components
– WCS provides specialized interface for geospatial data
• OWS-9 WCS for LIDAR
– Implementation of WCS 2.0 service that serves the LIDAR data in
NITF format
– service supports the JPEG 2000 output format along with GMLJP2
metadata description as well as the JPIP protocol to deliver the
output JPEG2000 data.
– Demonstrated WCS with JPIP streaming for GMLJP2.
OGC ®
OWS-9 Coverage Access ER (Editor P. Bauman, OGC 12-095)
OWS-9 OWS Innovations WCS for LIDAR Engineering Report (Editor W. Han et.al., OGC 12-155)
© 2014 Open Geospatial Consortium 22

23. OGC ®
OGC Testbed 9:
LiDAR Geoprocessing with OGC Standards
OWS-9 OWS Innovations WCS for LIDAR Engineering Report (Editor W. Han et.al., OGC 12-155)
© 2014 Open Geospatial Consortium 23

24. Performance of OGC Services in the Cloud
• OGC service standards have been widely adopted and
deployed across the world but little information has filtered
out regarding the performance and scalability of products
based on OGC standards.
• Easy access to large IT computing resources such as the
Amazon EC2 Cloud infrastructure provides an opportunity
to use a flexible and low cost IT resource environment to
investigate the performance and scalability of products
based on OGC standards.
• OGC Testbed 10 tested web mapping and other geo-processing
OGC ®
use cases as a way to characterize the
performance of OGC data services deployed in Cloud
infrastructures.
© 2014 Open Geospatial Consortium 24

25. Concurrent user access to OGC-compliant map servers using an
Amazon EC2 with an S3 network attached storage configuration
OGC ®
OGC Testbed 10:
WMTS Massively Available with Cloud Services
© 2014 Open Geospatial Consortium

26. OGC ®
OGC Testbed 10:
Performance & Scalability of WMS/WMTS
• Understand how
OGC services
perform on clouds
through testing in
OWS-10 Testbed
• Response time with
concurrent users for
– WMS: linear
– WMTS: flat to 1800
!
OGC Testbed 10 Performance of OGC Services in the Cloud ER (Editor Edric Keighan, OGC 14-028r1)

27. OGC ®
OGC Testbed 10:
SAR Interferometry with WPS on SBAS Cloud
• Performance enhancements with Cloud deployment of
SBAS (Small Baseline Subset) processing application
using WPS and OpenSearch OGC Web Services
© 2014 Open Geospatial Consortium 27
!
• Exploit 64 differential SAR scenes
for the generation of time series
showing ground displacements
over a decade in geological
sensitive areas.
• Part of an ongoing effort from
ESA, CNR-IREA and Terradue
partners

28. As a result of this testbed:
User organizations able to
request service providers (data
provider, algorithm provider) to
operate under a multi-tenant
Cloud environment with specific
performance constraints, in order
to build and run scientific
applications that deliver added-value
OGC ®
OGC Testbed 10:
Moving Cloud Processing to Data
1. An OGC WPS Service “bought” on
Terradue Cloud Marketplace
2. then deployed to Amazon
UK Cloud
3. which computes using
data from Interoute Cloud UK
Experiments led by Terradue in OGC Testbed 10
OGC Testbed 10 Performance of OGC Services in the Cloud ER (Editor Edric Keighan, OGC 14-028r1)
products.

29. OGC Testbed 11: Urban Climate Resilience
• WFS-T Transactions • data representation
• Desktop/mobile clients
• Geosynchronization
• REST and GeoJSON
OGC ®
• simulation models integration
• high-resolution data
• auxiliary data integration
Copyright © 2014 Open Geospatial Consortium
• What-if scenarios & models
• Social/political data (TJS)
• GeoPackage creation
• Georeferenced grid harmon.
• Streaming
• Alerting
• GeoPackage creation
• GeoPackage synchronization
• Image streaming tests
OGC Testbed 11 UCR Thread Architect: Dr. Ingo Simonis,
OGC Director Interoperability and Science

30. OGC ®
OGC Testbed 11 Threads
RFQ/CFP: http://www.opengeospatial.org/standards/requests/126
Geo4NIEM
• NIEM V3.0 IEPD
Geoprocessing
Round-Trip
• NIEM Enhancements
Security Tagging
• NIEM-GML Feature
Processing
• Security: Identification
Authorization, Access
• And more…
© 2014, Open Geospatial Consortium 30
Urban Climate
Resilience (UCR)
• Climate/Big Data
processing - WPS
• Hi-Res Flood Model
• Social Media
• WFS-T with REST
• GeoPackage, GeoSync
• Georeferenceable Grid
Harmonization
• And more…
Cross-Community
Interoperability (CCI)
• Aviation: NOTAM, AFX
• REST and SOAP
• JSON/GeoJSON
• Semantic Broker:
Social Media, Linked
Data, GeoSPARQL
• SPARQL for Symbology
• Compliance
• And more…
Cross Thread Scenario: Displaced Population Due to Coastal Inundation

31. Relevant OGC Engineering Reports
These engineering reports (ER) provide more detail on the topics addressed in
OGC Testbeds listed above:
• OWS-8 ER Metadata Mapping between NASA-EOS and WCS 2.0 (11-090)
• OWS-8 WCS 2.0 EO Application Profile ER (11-096)
• OWS-8 WCS 2.0 EO AP Compliance Tests and Reference Implementation ER
(11-095)
• OWS-8 Geoprocessing of EO (11-116)
• OWS-9 Map Tiling Methods Harmonization (12-157)
• OWS-9 Data Quality for Web Mapping ER (12-160)
• OWS-9 Coverage Access ER (12-095)
• OGC Testbed 10 Service Integration ER (14-013)
• OGC Testbed 10 Performance of OGC Services in the Cloud ER (14-028r1)
OGC ®
© 2014 Open Geospatial Consortium 31

32. OGC ®
For Details on OGC Standards…
• OGC Standards
– Freely available
– www.opengeospatial.org/standards
• OGC Reference Model (ORM)
– Overview of OGC Standards Baseline
– Resource for defining architectures for specific
applications
– www.opengeospatial.org/standards/orm
George Percivall, gpercivall at opengeospatial.org