Pedro Gonçalves
Rolling out
Terradue Cloud Platform
ISRSE-37, Tshwane, South Africa 8-12 May 2017
What does a Platform mean nowadays?
▪ A lever of Web and Cloud technologies
▪ A business model for value co-creation
▪ A framework to bring innovation to new or
larger communities
Platform
▪ A workplace to support the exploitation of
Earth Observations by value adding partners
▪ A laboratory to develop algorithms and deploy
compute-intensive campaigns
▪ A full suite of enablers to support capacity
building efforts
▪ A set of community-oriented knowledge bases
Earth Observation Platform
▪ EO data is a unique source of global
measurements over decades
▪ When calibrated and combined with other sources, it
empowers validation & interpolation models
▪ Collaboration process: a feedback
loop of providers & consumers
▪ More providers of pre-processed data layers
▪ New demands for structured information
The growth model is pulled by the communities
Earth Observation Platform
Producers
Consumers
▪ Integration layers:
▪ Connecting scientists to the Web and Cloud resources
empowering their research
▪ Tailored for Earth Sciences data integration & analysis
▪ Partnership mindset for interplay
between Data Providers, Developers,
Integrators and Producers
▪ Dedicated entry points for different types of partners
Earth Observation PlatformD
evelopers
Producers
D
ata
Providers
Integrators
• Terradue started as an
ESA spin-off in 2006
• A common vision
co-funded by ESA &
EC projects
• Evolutions validation
via user engagement
• Network effects
carrying business
opportunities
From where we come
2009-2014
2015-Forward
ESA G-POD
G-POD Cloud
Experiment
GEO Supersites
Virtual Archive
EO Thematic
Platforms
GENESI-DR/DEC
GEOWOW
MELODIES
H2020 Projects
2006-2008
▪ Ellip is the new Terradue Cloud Platform
▪ A Workplace with APIs fostering partnerships
with Providers, Developers, Integrators and
Information Producers for:
▪ Rapid prototyping and benchmarking
▪ Seamless data access whatever stage of the process
▪ Automated data processing
▪ Scalable resource management
▪ Information conveyance
Mission
• Data Agency
Data Discovery and
Access Via Standard
Protocols
• Cloud Controller
Cloud Deployments
Ready to Scale Out to
several Cloud providers
ICT and Data Resources
Distributed
Data Sources
VA4 (ESA / GEO)
SciHub (ESA)
CSCDA (EC / ESA)
PEPS (CNES)
ASF (NASA)
Google Cloud
Amazon Cloud
…
• An infrastructure ready
to manage configurations
& automate provisioning
• A DevOps process with
proven software
development practices
for code versioning,
automated testing and
continuous delivery
Infrastructure as Code
Where the
Application
code lives
Defines and
automates the
Application
build phases
Triggers and
controls the
Application
builds and
deploys the
outputs
Where the
Application
package(s)
are stored
Maven Jenkins ArtifactoryGithub
• Apps integration
• C/C++, Java, Python
• Matlab and IDL
• Lifecycle management
using RPMs and GitHub
• Apps directly integrated on
Community Portals and
accessed through API
Integrator Frontends
• Evaluating service
metrics and controlling
usage expectations and
performance trends.
• Yield trends useful to
detected problems and
provide guidance.
Platform & Service Metrics
• Business model for
sustainability of hosted
processing operations
• Public funding, commercial
revenues, and costs-sharing
• Cloud providers
• Processors developers
• Applications Integrators
• Scientific users
Subscription Plans
EXPLORER
SUBSCRIPTION
PLAN
Low cost
App. development
For
Data science users
SCALER
SUBSCRIPTION
PLAN
Low cost
Cluster processing
For
Producer users
PREMIUM
PARTNER
SUBSCRIPTION PLAN
Higher cost
Innovation
& platform evolutions
For
Integrator users
4 Partner Programs
◆ Technology Alliance
◆ Virtual Archive
◆ Data Science
◆ Cloud Appliance
Hosted Communities
↗
H2020 ECOPOTENTIAL
(2015-2018)
↗
H2020 EVER-EST
(2015-2018)
↗
H2020 Co-ReSyF
(2016-2018)
↗
H2020 Copernicus App Lab
(2016-2018)
↗
H2020 NextGEOSS
(2016-2020)
↗
H2020 INTAROS
(2016-2021)
ESA Exploitation Platforms
(2014 onwards)
35+
EC FP7 MELODIES
(2014-2016)
14
EC FP7 SenSyF
(2013-2015)
8
EC FP7 GEOWOW
(2011-2013)
5
Processing Campaigns
(2013-onwards)
30+
▪ Climate Change
▪ Carbon market
▪ Oceanography
▪ Climatology
▪ Biodiversity
▪ Volcanology
▪ Ship logistics
▪ Water quality
▪ Desertification
▪ Seismic analysis
▪ Urban accounting
▪ Land management
▪ Disaster Risk Reduction
Hosted Communities
↗
H2020 ECOPOTENTIAL
(2015-2018)
↗
H2020 EVER-EST
(2015-2018)
↗
H2020 Co-ReSyF
(2016-2018)
↗
H2020 Copernicus App Lab
(2016-2018)
↗
H2020 NextGEOSS
(2016-2020)
↗
H2020 INTAROS
(2016-2021)
ESA Exploitation Platforms
(2014 onwards)
35+
EC FP7 MELODIES
(2014-2016)
14
EC FP7 SenSyF
(2013-2015)
8
EC FP7 GEOWOW
(2011-2013)
5
Processing Campaigns
(2013-onwards)
30+
▪ Climate Change
▪ Carbon market
▪ Oceanography
▪ Climatology
▪ Biodiversity
▪ Volcanology
▪ Ship logistics
▪ Water quality
▪ Desertification
▪ Seismic analysis
▪ Urban accounting
▪ Land management
▪ Disaster Risk Reduction
• DLR’s InSAR processing chain
automated for all S-1 pairs
over world tectonic areas
• Amplitude, Coherence &
interferograms for all 6- and
12-day pairs. About 100 / day
browse images at 100m
Systematic Processing
Examples of DEM corrected inteferograms
Coherence (R) and
Amplitude (G, B)
over la Réunion
Collection of
coherence &
amplitude
composites over
Europe
• High-Resolution Change
Monitoring of Volcanoes
& Corinth Rift
Laboratory (EPOS Near
Fault Obs.)
• Processing with the DLR
High-Resolution InSAR
(50m-25m) of S-1 pairs
Systematic Processing
• Flood extent service:
Detected by SAR to a
reference mask with a
regular water content
scenario.
• Flood risk Service:
Affectation in the sensitive
areas present
• Statistics
Frequency maps, floodable
area and time series.
Systematic Processing
• Sentinel-2 data staging
for specific locations
• Deploying atmospheric
correction code from
GitHub to production
• Results registered via
Zenodo to create a DOI
(citation record)
Systematic Data Staging
▪ Last message : we’re hiring!
▪ Join our team in Rome La Città Eterna
15 members, 5 EU nationalities
▪ We are looking for experts in EO
applications and cloud technologies
▪ Exciting new challenges rolling out Ellip
for Earth Science
That’s it !
Contact us !
pedro@terradue.com

ISRSE37 Terradue Cloud Platform & ellip

  • 1.
    Pedro Gonçalves Rolling out TerradueCloud Platform ISRSE-37, Tshwane, South Africa 8-12 May 2017
  • 2.
    What does aPlatform mean nowadays? ▪ A lever of Web and Cloud technologies ▪ A business model for value co-creation ▪ A framework to bring innovation to new or larger communities Platform
  • 3.
    ▪ A workplaceto support the exploitation of Earth Observations by value adding partners ▪ A laboratory to develop algorithms and deploy compute-intensive campaigns ▪ A full suite of enablers to support capacity building efforts ▪ A set of community-oriented knowledge bases Earth Observation Platform
  • 4.
    ▪ EO datais a unique source of global measurements over decades ▪ When calibrated and combined with other sources, it empowers validation & interpolation models ▪ Collaboration process: a feedback loop of providers & consumers ▪ More providers of pre-processed data layers ▪ New demands for structured information The growth model is pulled by the communities Earth Observation Platform Producers Consumers
  • 5.
    ▪ Integration layers: ▪Connecting scientists to the Web and Cloud resources empowering their research ▪ Tailored for Earth Sciences data integration & analysis ▪ Partnership mindset for interplay between Data Providers, Developers, Integrators and Producers ▪ Dedicated entry points for different types of partners Earth Observation PlatformD evelopers Producers D ata Providers Integrators
  • 6.
    • Terradue startedas an ESA spin-off in 2006 • A common vision co-funded by ESA & EC projects • Evolutions validation via user engagement • Network effects carrying business opportunities From where we come 2009-2014 2015-Forward ESA G-POD G-POD Cloud Experiment GEO Supersites Virtual Archive EO Thematic Platforms GENESI-DR/DEC GEOWOW MELODIES H2020 Projects 2006-2008
  • 7.
    ▪ Ellip isthe new Terradue Cloud Platform ▪ A Workplace with APIs fostering partnerships with Providers, Developers, Integrators and Information Producers for: ▪ Rapid prototyping and benchmarking ▪ Seamless data access whatever stage of the process ▪ Automated data processing ▪ Scalable resource management ▪ Information conveyance Mission
  • 8.
    • Data Agency DataDiscovery and Access Via Standard Protocols • Cloud Controller Cloud Deployments Ready to Scale Out to several Cloud providers ICT and Data Resources Distributed Data Sources VA4 (ESA / GEO) SciHub (ESA) CSCDA (EC / ESA) PEPS (CNES) ASF (NASA) Google Cloud Amazon Cloud …
  • 9.
    • An infrastructureready to manage configurations & automate provisioning • A DevOps process with proven software development practices for code versioning, automated testing and continuous delivery Infrastructure as Code Where the Application code lives Defines and automates the Application build phases Triggers and controls the Application builds and deploys the outputs Where the Application package(s) are stored Maven Jenkins ArtifactoryGithub
  • 10.
    • Apps integration •C/C++, Java, Python • Matlab and IDL • Lifecycle management using RPMs and GitHub • Apps directly integrated on Community Portals and accessed through API Integrator Frontends
  • 11.
    • Evaluating service metricsand controlling usage expectations and performance trends. • Yield trends useful to detected problems and provide guidance. Platform & Service Metrics
  • 12.
    • Business modelfor sustainability of hosted processing operations • Public funding, commercial revenues, and costs-sharing • Cloud providers • Processors developers • Applications Integrators • Scientific users Subscription Plans EXPLORER SUBSCRIPTION PLAN Low cost App. development For Data science users SCALER SUBSCRIPTION PLAN Low cost Cluster processing For Producer users PREMIUM PARTNER SUBSCRIPTION PLAN Higher cost Innovation & platform evolutions For Integrator users 4 Partner Programs ◆ Technology Alliance ◆ Virtual Archive ◆ Data Science ◆ Cloud Appliance
  • 13.
    Hosted Communities ↗ H2020 ECOPOTENTIAL (2015-2018) ↗ H2020EVER-EST (2015-2018) ↗ H2020 Co-ReSyF (2016-2018) ↗ H2020 Copernicus App Lab (2016-2018) ↗ H2020 NextGEOSS (2016-2020) ↗ H2020 INTAROS (2016-2021) ESA Exploitation Platforms (2014 onwards) 35+ EC FP7 MELODIES (2014-2016) 14 EC FP7 SenSyF (2013-2015) 8 EC FP7 GEOWOW (2011-2013) 5 Processing Campaigns (2013-onwards) 30+ ▪ Climate Change ▪ Carbon market ▪ Oceanography ▪ Climatology ▪ Biodiversity ▪ Volcanology ▪ Ship logistics ▪ Water quality ▪ Desertification ▪ Seismic analysis ▪ Urban accounting ▪ Land management ▪ Disaster Risk Reduction
  • 14.
    Hosted Communities ↗ H2020 ECOPOTENTIAL (2015-2018) ↗ H2020EVER-EST (2015-2018) ↗ H2020 Co-ReSyF (2016-2018) ↗ H2020 Copernicus App Lab (2016-2018) ↗ H2020 NextGEOSS (2016-2020) ↗ H2020 INTAROS (2016-2021) ESA Exploitation Platforms (2014 onwards) 35+ EC FP7 MELODIES (2014-2016) 14 EC FP7 SenSyF (2013-2015) 8 EC FP7 GEOWOW (2011-2013) 5 Processing Campaigns (2013-onwards) 30+ ▪ Climate Change ▪ Carbon market ▪ Oceanography ▪ Climatology ▪ Biodiversity ▪ Volcanology ▪ Ship logistics ▪ Water quality ▪ Desertification ▪ Seismic analysis ▪ Urban accounting ▪ Land management ▪ Disaster Risk Reduction
  • 15.
    • DLR’s InSARprocessing chain automated for all S-1 pairs over world tectonic areas • Amplitude, Coherence & interferograms for all 6- and 12-day pairs. About 100 / day browse images at 100m Systematic Processing Examples of DEM corrected inteferograms Coherence (R) and Amplitude (G, B) over la Réunion Collection of coherence & amplitude composites over Europe
  • 16.
    • High-Resolution Change Monitoringof Volcanoes & Corinth Rift Laboratory (EPOS Near Fault Obs.) • Processing with the DLR High-Resolution InSAR (50m-25m) of S-1 pairs Systematic Processing
  • 17.
    • Flood extentservice: Detected by SAR to a reference mask with a regular water content scenario. • Flood risk Service: Affectation in the sensitive areas present • Statistics Frequency maps, floodable area and time series. Systematic Processing
  • 18.
    • Sentinel-2 datastaging for specific locations • Deploying atmospheric correction code from GitHub to production • Results registered via Zenodo to create a DOI (citation record) Systematic Data Staging
  • 19.
    ▪ Last message: we’re hiring! ▪ Join our team in Rome La Città Eterna 15 members, 5 EU nationalities ▪ We are looking for experts in EO applications and cloud technologies ▪ Exciting new challenges rolling out Ellip for Earth Science That’s it !
  • 20.