Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Capacity building, validation and repeatability

58 views

Published on

BlueBRIDGE workshop: "Supporting Blue Growth with innovative applications based on EU e-infrastructures" - Brussels February 2018

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Capacity building, validation and repeatability

  1. 1. BlueBRIDGE receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 675680 www.bluebridge-vres.eu Capacity building, validation and repeatability Pasquale Pagano BlueBRIDGE Technical Director pasquale.pagano@isti.cnr.it Supporting Blue Growth with innovative applications based on EU e-infrastructures 14-15 February 2018, Brussels
  2. 2. BlueBRIDGE Services Offer Tailored services addressing three heterogeneous domains One common infrastructure BlueBRIDGE Infrastructure 2Capacity building, validation and repeatability
  3. 3. BlueBRIDGE Capacities Capacity building, validation and repeatability Data services addressing the whole data management value chain Infrastructure services addressing the access and management of computing and storage resources 3
  4. 4. BlueBRIDGE hides ICT complexities Capacity building, validation and repeatability Storage Database GIS NoSQL Computing D4Science EGI SeaDataNet Data OSCAR Copernicus 4
  5. 5. BlueBRIDGE: Figures Users Distribution, 2017 BlueBRIDGE is Connecting 3000 scientists operating with 99,8% service availability BlueBRIDGE hosts +50 Virtual Laboratories to deliver capacities for the aquaculture, biodiversity, fisheries, marine spatial planning, and stock assessment communities world-wide https://bluebridge.d4science.org/explore Users VLabs Capacity building, validation and repeatability
  6. 6. BlueBRIDGE: Figures Highlights Connecting 3000 scientists Capacity building, validation and repeatability 6
  7. 7. the ability of a service or service component to fulfil its intended function at a specific time, excluding scheduled maintenance periods BlueBRIDGE: Figures Highlights 28 hours down 895 days up the ability of a server to fulfil its intended function at a specific time operating with 99,47% service reliability 3 hours critical 45 days up 2626 services operating with 99,87% availability Capacity building, validation and repeatability 7
  8. 8. VRE Management Resource Catalogue Management Spatial Data Infrastructure Management Collaborative Environment Management Data Analytics Building on consolidated capacities Capacity building, validation and repeatability 8
  9. 9. VRE Management framework A wizard-based mechanism easing the production of a VRE specification (functionalities, datasets, membership policy) A service consuming the VRE Specification to create the VRE deployment plan and execute it It allows to easily access to several existing technologies, e.g. Rstudio, Rshiny, ShareLatex, PostgreSQL, SDI, … It enables user management facilities to support per-VRE authentication, authorization, accounting, and quota VRE Management Capacity building, validation and repeatability
  10. 10. A service to publish, search & browse catalogue items Items are URI based metadata + resources Ready for item typologies customisation (e.g. fields, field values, tagging and browsing directives) Multi-tenancy and VRE- friendly Based on standards, e.g. OGC for spatial data and SDMX for statistical data Resource Catalogue Capacity building, validation and repeatability
  11. 11. A standard-based, scalable, robust infrastructure for managing spatial data Catalogue (CSW) Repositories (WCS, WMS, WFS) Analytics (WPS) Fully-integrated with Authentication, Authorization, and Accounting REST-based access to SDI services Multi-tenancy and VRE-friendly Spatial Data Infrastructure Capacity building, validation and repeatability
  12. 12. A virtual workspace resembling a file manager Items range from binary files to complex objects VRE Folders to share items with VRE Members Automatic versioning and business metadata Integrated with publishing (into the resource catalogue, into the SDI) Integrated with analytics (read / write) A social networking environment to post messages, links, and files by users and applications reply to posts, subscribe to post discussions hash tags (#) to organize messages mentions (@) VRE Members / Groups E-mail notification Collaborative Environment Capacity building, validation and repeatability
  13. 13. Not another cloud computer platform but a platform where executions can be repeated, compared, discussed, logged Not another computational engine but a platform where interdisciplinary tools and services can be easily contributed by the communities Data Analytics Rich-array of ready-to-use algorithms Extensible with respect to supported algorithms, programming “languages” and models and enactment platforms (hybrid model) Algorithms integration by annotations Algorithms reuse and repurpose, process repeatability (PROV-O) Per-algorithm GUI and programmatic access by WPS Capacity building, validation and repeatability
  14. 14. Capacity building, validation and repeatability Script provider Updates the script on his private Workspace The BB service configures the infrastructure on-the-fly A user executes an experiment on his/her data The output, the input and the parameters can be shared with another user This user can execute the experiment again and share the computation with other users 1 2 3 4 5 6 7 8 9 5 BlueBRIDGE enables new scientific workflows 14
  15. 15. Conclusions • simplifies the integration of user applications into e- infrastructure • promotes access to an integrated environment where personal applications are managed as enabling technologies • ensures Reuse, Repeatability, Reproducibility transparently and horizontally across all services • delivers a community-cloud where virtual services hide cloud and related technologies • delivers a single access point to system of systems Capacity building, validation and repeatability manages supports operates User exploits Policies Authz ToU Training Adoption Use Monitoring Accounting Provision Security Discover Access Re-use Share 15
  16. 16. Questions and Answers Time BlueBRIDGE Gateway https://bluebridge.d4science.org Capacity building, validation and repeatability 16
  17. 17. Contact details Visit the BlueBRIDGE Website: http://www.bluebridge-vres.eu Visit the BlueBRIDGE Catalogue & VREs: https://bluebridge.d4science.org/catalogue Capacity building, validation and repeatability 17
  18. 18. Infrastructure for Geeks • uses cloud-computing technologies to manage +300 servers (more than 5 TB Ram, +1700 CPUs, 400 TB storage) Capacity building, validation and repeatability 18 • is monitored via Nagios and Ganglia • is maintained via Ansible (Application Deployment + Configuration Management + Continuous Delivery) • is governed by deployment and operation policies • has established SLA • is operated according to defined Terms of Use

×