Presentation by Fernando Aguilar Gómez (Instituto de Fisica de Cantabria) at the Delft3D User Days, during Delft Software Days 2016. Tuesday 1 November 2016, Delft.

1. Calibration and scenario generation of
Hydrodynamics and Water Quality
models over a Cloud Computing
environment: INDIGO-DataCloud
RIA-653549
Presented by Fernando Aguilar (IFCA-CSIC)
aguilarf@ifca.unican.es
INDIGO-DataCloud WP2
Delft3D Days – 1st November 2016

2. Introduction
• Framework: Collaboration with SME Ecohydros. Last year ROEM+.
• Reservoir Hydrodinamic and Water Quality modelling. Cuerda del Pozo: water supply,
water activities.
• Previous work
• Platform takes data from water: physical, chemical, biological, etc. Allows to know water status (data
taken since 2010 aprox.)
• Data visualization tool. Aims to alert authorities when the water quality is under the limits.
• High Resolution Models require a powerful and scalable computing environment.
November 2016 2

3. Case Study: Algae Bloom in a Water Reservoir
• Research Community: LifeWatch (ESFRI)
• Topic/Area: Biodiversity & Ecosystem research
• Objective of the Case Study:
• Monitor the evolution of the potential eutrophication of a Water Reservoir.
• Two main objectives:
• Scenarios: What if? Water flow, nutrients, etc. Range.
• Calibration: Algae Parameters like Mortality, Respiration, etc.
• Both requires a number of simulations.
• Scalability and flexibility needed.
• User Friendly Management: Input, Configuration, N outputs.
• Cloud Computing is the answer!
• INDIGO-DataCloud Project is developing all the solution needed.
3

4. The INDIGO-DataCloud Project
• INDIGO = INtegrating Distributed data Infrastructures for Global ExplOitation
• H2020 project, Apr 2015 – Oct 2017, 11M
EURO
• 26 partners in 11 European countries
• Objectives: Develop a data/computing
platform, targeting various scientific
communities and deployable on hybrid
(private or public) Cloud infrastructures
• Website: https://www.indigo‐datacloud.eu
INDIGO-DataCloud RIA-653549
4

5. INDIGO-DataCloud RIA-653549
INDIGO
Architecture
5
• Enhanced features in
• IaaS
• PaaS
• SaaS
• 1st release MidnightBlue
• Data Center Solutions
• Data Solutions
• Automated Solutions
• High-level User Oriented
Solutions

6. INDIGO Communities
November 2016 6
Life Sciences: ELIXIR, INSTRUCT/WeNMR, EuroBioImaging
Physical Sciences & Astronomy: CTA, LBT, WLCG
Social Sciences & Humanities: DARIAH, DCH-RP
Environmental Science: LifeWatch, EMSO, ENES
Agile Methodology to satisfy User Requirements!

7. Solution Developed
7
Local OneClient

8. Solution Developed
8
• Access to all the services
• Roles/Groups Definition
• Federated

9. Solution Developed
9
• Distributed Storage Solution
• Accessible by users and
Machines
• DropBox-like, but online
• WebClient, POSIX, etc.
• Sharable Environment
• Input/Output

10. Solution Developed
10
• GitHub: Software Repository. Script for running the “job”.
• DockerHub: SO, Delft3D Software and all dependencies ready to be
deployed.
• Ansible: Recipes for installing requirements (OneData).

11. Solution Developed
11
• User Interface
• Generates TOSCA template
from user configuration,
including model params.
• Sends json to Orchestrator.
• Everything transparent for
the user.

12. Solution Developed
12
• PaaS Orchestrator: Deploys the docker and manage the execution.
• Deployment over many different IaaS Services.
• Using Mesos/Chronos.
• Running instances mount OneData, get the input and write the output.
• Output available for the user.

13. Solution Developed
13
Local OneClient

14. Demo description
• Testbed resources that will be used: Bari TestBed
• Teams involved: INFN/Bari, PSNC, IFCA/CSIC
• Prerequisites: application in Docker
• Sequence of actions
• Connect to IAM to access OneData. Input data upload.
• Access to the Graphical User Interface (FutureGateway). Fill the form (OneData,
Access, Sweep Parameter values). Submit.
• The TOSCA template edited and sent to the orchestrator.
• Check Deployment status.
• After finishing, output accessible via OneData. Comparing models using Delft3D
tools.
• NOW WE SWITCH TO THE DEMO SCREEN…
14

15. INDIGO added value
• Scalable (storage and computing) resources in the cloud to perform
o(100-10000) tests…
• …and share directly within the community
• User Friendly interface to use cloud resources:
• Final users only need to fill a form to submit a new simulation, avoiding
the script edition or direct contact with the infrastructure
(Supercomputer, Grid, Cloud) (very helpful for non IT experts).
• First time we use a flexible and “universal” user authentication (quite
relevant to collaborate with SMEs also)
• Transparent access to shared large storage (OneData)
15

16. Conclusions
• Cloud Computing environment is a recommendable
framework to run a number of Delft3D simulations.
• INDIGO-DataCloud solutions integration offers a user-
oriented alternative: powerful and easy-to-use.
• IT skills are not needed to use Computing Resources.
• Distributed and Sharable Storage system available: no
need to copy files and scientists can collaborate easily.
• AAI Solution for accessing.
• INDIGO-DataCloud solutions available for using (Open
Source).
16

17. Thank you
https://www.indigo-datacloud.eu
Better Software for Better Science.
17
Fernando Aguilar (IFCA-CSIC)
aguilarf@ifca.unican.es