Big Data from Space outreach presentation describing the "Bring User to Data Scenario" and the Research and Service Support provided for scientific users, research centres, small and medium size enterprises, industry and institutions.

1. ESA Unclassified - For Official Use
Big Data from Space
Research, Technology and Innovation

2. ESA UNCLASSIFIED – For Official Use
We are surrounded by BIG DATA
Textual
information
(Google crawlers)
Geo-location
(Google Earth, Google
Maps)
Sharing multimedia
Social Networks
Credits: Psdblast.com

3. ESA Unclassified - For Official Use
Big data from Space

4. ESA Unclassified - For Official Use
Big data from Space: VOLUME
Sheer value of sensed data: archived data is currently
reaching the exabytes scale.

5. ESA Unclassified - For Official Use
Big data from Space: VELOCITY
New data is acquired almost on a continuous basis and
with an increasing rate

6. ESA Unclassified - For Official Use
Big data from Space: VARIETY
Data is delivered by sensors acting over various frequencies
of the electromagnetic spectrum in passive and active
modes

7. ESA Unclassified - For Official Use
Big data from Space: VERACITY
Sensed data is always associated with uncertainty and
accuracy measurements

8. ESA Unclassified - For Official Use
Big data from Space: VALUE
Our capacity to extract information and meaning from data

9. ESA Unclassified - For Official Use
Big data from Space: focus on USERS
Various space-related “domains” are concerned: Earth Observation, Space
Science, Solar System Objects, Space Situational Awareness, Secure Societies, etc.

10. ESA Unclassified - For Official Use
Overview
Data Lifecycle
Infrastructures and Services for Big Data
Big Data from Space Scenarios
Objectives and Activities

11. ESA UNCLASSIFIED – For Official Use
Big Data Lifecycle
Acquire Organize
Information
Provision
Analyse

12. ESA UNCLASSIFIED – For Official Use
Big Data Lifecycle:
Supporting Resources
SUPPORTING RESOURCES AND INFRASTRUCTURES
Architectures
Data selection &
pre-processing
Data & Software
Frameworks
Processing
Algorithms
Provenance &
Security

13. ESA Unclassified - For Official Use
Big Data from Space:
Technology Themes
Examples
Data Lifecycle
Processing and Analysis
Visualization and Visual Analytics
Multi-temporal Analysis
Onboard Computing, Compression and Transmission
Quality, Provenance and Trust
Infrastructures and Services for Big Data
Storage and computing platforms
Interoperability and Standards
Heterogeneous Data Sources (incl. auxiliary and collateral data)
Linked Data and Semantics
Data Openness, Privacy and Security
Software Defined Networking
User Management and Access to Resources

14. ESA Unclassified - For Official Use
Scenario 1: Space Science
Limited downlink capabilities
Data Volumes and Variety
Transmission on huge distances
Use of landers
Use of relay satellites

15. ESA Unclassified - For Official Use
Scenario 1: Space Science
Ground Segment
Big Data Stream Big Data Analysis
Processing &
Triage
Scalable
Archiving
Distributed
Archives
Data
Analysis
Science Team
Research
Community
Data Models &
Simulations
Cal/Val Activity

16. ESA Unclassified - For Official Use
Scenario 2: Earth Observation
Satellite constellations
Nano Satellites, RPAs,
etc.
Satellite instruments
Relay satellites

17. ESA Unclassified - For Official Use
Scenario 2: Earth Observation
Ground Segment
Big Data Stream Big Data Analysis
Cal/Val Activity
Validation Research
Community
Data Models
simulations
Applications
Satellite constellations
Nano Satellites, Drones, etc.
Satellite instruments
Services

18. ESA Unclassified - For Official Use
Scenario 3: Space and Security
Satellite Data
Aerial, In-Situ and Collateral Data
Decision Makers
First Responders
Operational
Information
Support EU external
security policies
Border SecurityResilience
against Disasters
Protection of Critical
Infrastructures

19. ESA UNCLASSIFIED – For Official Use
Scenario 4 : Big Data Exploitation
Current: “bring data to users”
Data
Ground Segment User Segment
Mission
Operator
User
Processing
resources
Data
Request
Discussion
New results
Long feedback loop
Data
Download
Communities

20. ESA UNCLASSIFIED – For Official Use
Scenario 4 : Big Data Exploitation
Future: “bring users to data”
Ground Segment User Segment
Mission
Operator
User
Discussion
Processing
resources
New results
Data
Request & Processing
Data
Shorter feedback loop
Communities

21. ESA Unclassified - For Official Use
Big Data from Space: Objectives
1. Identification of priorities and implementation of a plan for
research, technology development and innovation
2. Widening competences and expertise of universities, research
institutes, labs, SMEs and industrial actors in the field
3. Foster networking of experts and users towards better access
and sharing of data, tools and resources
4. Leverage innovation, spin-in / spin-off of technologies and
business development arising from research and industry
progress
5. Increase and promote the value stemming from the huge
quantity of data made available nowadays (and in the future)
6. Contribute to the EO innovation for Europe, as one of the
main pillars for the Ground Segment evolution strategy

22. ESA UNCLASSIFIED – For Official Use
The Big Data from Space initiative shall:
1. Foster and coordinate the European Research & Development activities to
address forthcoming Big Data challenges
2. Define and implement technology development programmes supporting cross-
cutting enabling technologies in the Big Data field
3. Establish and promote a close and permanent dialogue with key industry and
SMEs actors
4. Contribute to the exploration and implementation of “partnership models” for
the actual exploitation of future Big Data flows
5. Contribute to the EO innovation for Europe, as one of the main pillars for the
Ground Segment evolution strategy
6. Support academic activities to build a new figure: the data scientist
Recommendations

23. ESA UNCLASSIFIED – For Official Use
BiDS Activities in EO / EOP-G
1. Research and Service Support (RSS)
Since 2007, RSS - as part of ESA EO Ground Segment Department -
provides tools and services to support the user communities in EO data
exploitation (http://wiki.services.eoportal.org/tiki-custom_home.php):
• G-POD (Grid Processing on Demand): operational since 2007
- operational processing environment
- access to available EO products
- integration of specific applications/processors
- coupled with high-performance and sizeable computing resources
• Cloud Toolbox
- customized Virtual Machines (VMs) deployed on a cloud infrastructure
- fast access to the ESA RSS Data Farm archive.
- access through user devices (pc, laptop, tablet) from anywhere

24. ESA UNCLASSIFIED – For Official Use
BiDS Activities in EO / EOP-G
2. Exploitation Platforms (EP)
Virtual workspaces providing user communities with access to:
a) large volumes of data (EO, auxiliary, in-situ)
b) algorithm development and integration environment
c) processing software and services (e.g. toolboxes, visualization tools)
d) computing resources (e.g. hybrid cloud/grid)
e) collaboration tools (e.g. forums, wiki, KnowledgeBase, social networks)
f) general operation capabilities (UM, AA, accounting)
 Thematic Exploitation Platforms: under development (Geohazards
[pre-operational qualification on-going], Urban, Forestry, Polar,
Hydrology, Coastal)
 Mission Exploitation Platforms (Proba-V, etc.)

25. ESA UNCLASSIFIED – For Official Use
BiDS Activities in EO / EOP-G
2. Generic GS functions in support of Big data management
Generic functions under development and pre-ops:
a) Federated User Management (SSO in operations, pilot completed,
generic component under development)
b) Data Hub Software (DHuS): open source component used in the frame
of Copernicus/Sentinels for data access management including data
hub relays (sentinels.copernicus.eu)

26. ESA UNCLASSIFIED – For Official Use
BiDS Activities in EO / EOP-G
4. Technology Projects (TRP/GSTP)
Identify requirements and develop relevant technologies in support to the
EO Ground Segment:
a) Harmonization and Standards (HMA, SMARTHMA)
b) EO Ontology (OTEG, RARE, SMAAD, OBEOS, ProdTrees)
c) EO Image Information Mining (KIM, KEO, ASIM, EOLib, PFA, OSIRIDE,
DAMATS)
d) Exploitation Platforms demonstrators (EP4SM, EO-RCP)
e) E-Collaboration platforms (E-CEO)

27. ESA Unclassified - For Official Use
Big Data from Space
Big Data from Space Conference - BiDS’16
15-17 March 2016, Tenerife, Spain!

28. ESA UNCLASSIFIED – For Official Use
BiDS’16 Organising Committee
J. Amador Monteverde ESA,
C. Arviset ESA,
S. Albani SatCen,
S. Baillarin CNES,
P. Baumann Jacobs University,
L. Bruzzone University of Trento,
M. Datcu DLR,
M. Iapaolo ESA,
J. Knapen IAC,
P.G. Marchetti ESA,
P. Soille JRC,
J.L. Valero SatCen

29. ESA Unclassified - For Official Use
Big Data from Space