Successfully reported this slideshow.

Pathways for EOSC-hub and MaX collaboration

1

Share

Loading in …3
×
1 of 22
1 of 22

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Pathways for EOSC-hub and MaX collaboration

  1. 1. Pathways for EOSC-hub and MaX collaboration A platform for reproducible science with full provenance Giovanni Pizzi
 giovanni.pizzi@epfl.ch 
 Theory and Simulation of Materials, EPFL Lausanne
  2. 2. The MaX European Centre of Excellence Materials design at the Exascale Partners 14 partners with unique expertise in: • Materials Science (CEA, CNR, EPFL, ETHZ, FZ Jülich, ICN2, SISSA) • Software development and code validation
 (AiiDA, BigDFT, CP2K, Fleur, Quantum ESPRESSO, SIESTA, Yambo) • HPC (5 TIER-0 HPC Centres: BSC, CEA, CINECA, CSCS, JSC) • Technology (ARM, E4) • Communication & Outreach (ICTP, Psi-K, CECAM, UGent, TRUST-IT) Key-Actions • Restructure MaX codes towards exascale and extreme scaling performance • Co-design activities for HPC architectures • Develop broader ecosystem enabling the convergence of HPC, HTC and HPDA (WP5) • Widen the access to codes and foster transfer of know-how to user communities Domains of interest • High-Performance Computing (HPC) • High-Throughput Computing (HTC) • High-Performance Data Analytics (HPDA) http://www.max-centre.eu
  3. 3. Leverage supercomputers to compute 
 and predict materials’properties In our context, high-throughput means 
 “10’000+ HPC simulations per day” Scientific aim: Compute properties for all 
 of them (and even new, invented ones)
 and discover novel functional materials
  4. 4. How to manage data, simulations and their provenance?
 IS THERE A REPRODUCIBILITY CRISIS? Nature 533
 452–454 (2016)
  5. 5. How to manage data, simulations and their provenance?
 IS THERE A REPRODUCIBILITY CRISIS? Nature 533
 452–454 (2016) 
 We need a tool to help us
 automate research, organise it, 
 store provenance guaranteeing reproducibility,
 then analyze results, 
 and finally share them and collaborate

  6. 6. MaX Centre of Excellence: AiiDA and Materials Cloud The challenges we address • Allow high-throughput research (10’000+ simulations/day); automate simulations, automatically track provenance • Share simulations according to the FAIR principles 
 and beyond, guaranteeing reproducibility • Encode scientists’knowledge in automated workflows
 (scientific know-how, numerical parameters, 
 choice of data to preserve and share) • Provide advanced data analytics tools • Provide HPC resources as services in an easy, accessible way to anybody, via simple interfaces
  7. 7. MaX Centre of Excellence: AiiDA and Materials Cloud Data provenance: Directed Acyclic Graphs G. Pizzi et al.,
 Comp. Mat. Sci. 111, 218-230 (2016) http://www.aiida.net MIT license (open source) Developed since 2013
 Used in production from many
 scientific research projects
  8. 8. “Simple”graphs of workflows for a single material KpointsData (216283) (372 kpts) MatdynCalculation (216285) FINISHED kpoints BandsData (216385) 'Phonon bands' output_phonon_bands FolderData (216383) retrieved RemoteData (216320) remote_folder ParameterData (216384) output_parameters Code (209961) 'matdyn-5.1.2-module' code ForceconstantsData (216273) parent_calc_folder ParameterData (216284) parameters ParameterData (216282) settings Q2rCalculation (215983) FINISHED force_constants FolderData (215981) parent_calc_folder Code (209960) 'q2r-5.1.2-module' code ParameterData (215982) parameters InlineCalculation (215980) recollect_qpoints_inline() retrieved FolderData (215920) retrieved_6 FolderData (215976) retrieved_7 FolderData (215978) retrieved_4 FolderData (215797) retrieved_5 FolderData (215848) retrieved_2 FolderData (215924) retrieved_3 FolderData (215831) retrieved_0 FolderData (215912) retrieved_1 FolderData (215895) retrieved_8 FolderData (215829) retrieved_9 FolderData (215252) initial_folder InlineCalculation (215309) distribute_qpoints_inline() retrieved PhCalculation (215430) FINISHED retrieved PhCalculation (215433) FINISHED retrieved PhCalculation (215526) FINISHED retrieved PhCalculation (215427) FINISHED retrieved PhCalculation (215520) FINISHED retrieved PhCalculation (215523) FINISHED retrieved PhCalculation (215517) FINISHED retrieved PhCalculation (215514) FINISHED retrieved PhCalculation (215436) FINISHED retrieved PhCalculation (215529) FINISHED retrieved PhCalculation (215224) FINISHED retrieved ParameterData (215428) settings RemoteData (214841) parent_calc_folder parent_calc_folderparent_calc_folder parent_calc_folder parent_calc_folder parent_calc_folder parent_calc_folder parent_calc_folder parent_calc_folder parent_calc_folder parent_calc_folder Code (209959) 'ph-5.1.2-module' code codecode code code code code code code code code ParameterData (215429) parameters KpointsData (215312) (1 kpts) qpoints ParameterData (215431) settings ParameterData (215432) parameters KpointsData (215313) (1 kpts) qpoints ParameterData (215524) settings ParameterData (215525) parameters KpointsData (215310) (1 kpts) qpoints KpointsData (215311) (1 kpts) qpoints ParameterData (215426) parameters ParameterData (215425) settings ParameterData (215518) settings ParameterData (215519) parameters KpointsData (215316) (1 kpts) qpoints ParameterData (215521) settings ParameterData (215522) parameters KpointsData (215317) (1 kpts) qpoints ParameterData (215516) parameters ParameterData (215515) settings KpointsData (215314) (1 kpts) qpoints ParameterData (215512) settings ParameterData (215513) parameters KpointsData (215315) (1 kpts) qpoints KpointsData (215318) (1 kpts) qpoints ParameterData (215435) parameters ParameterData (215434) settings ParameterData (215527) settings ParameterData (215528) parameters KpointsData (215319) (1 kpts) qpoints ParameterData (215223) parameters ParameterData (215222) settings KpointsData (214808) 4x4x4 (+0.0,0.0,0.0) qpoints PwCalculation (214830) scf FINISHED remote_folder qpoint_6 qpoint_7qpoint_4 qpoint_5 qpoint_2 qpoint_3 qpoint_0 qpoint_1 qpoint_8 qpoint_9 Code (139993) 'pw-5.1.2-module' code UpfData (1658) pseudo_O UpfData (1660) pseudo_Ti ParameterData (214828) parameters ParameterData (214829) settings KpointsData (214807) 6x6x6 (+0.0,0.0,0.0) kpoints UpfData (1905) pseudo_Ba StructureData (214814) BaO3Ti structure structure Phonon dispersion (atom oscillations around equilibrium positions: 
 thermal transport, electronic mobility, …) Molecular dynamics of Lithium in a solid electrolyte (Discover novel, safe and efficient electrolytes for Li- batteries) Elastic constants (response of materials to stresses and deformations)
  9. 9. Graphical representation of an AiiDA database of calculations and workflows
 of DFT band structure and Wannier functions
  10. 10. MaX Centre of Excellence: AiiDA and Materials Cloud An ecosystem of plugins https://aiidateam.github.io/aiida-registry/ 34 plugin entries for Materials Science, 
 supporting 65 codes, 62 workflows, …
  11. 11. MaX Centre of Excellence: AiiDA and Materials Cloud Open Science Platform: AiiDA + Materials Cloud https://www.materialscloud.org Online since February 2018 Cloud dissemination platform for FAIR data sharing
 and more (cloud simulation and data generation platform) + + …
  12. 12. MaX Centre of Excellence: AiiDA and Materials Cloud Open and FAIR data sharing: Archive, Discover, Explore Direct links
 to Discover &
 Explore DOIs
 assigned FAIRsharing.org
 re3data.org + Recommended
 data repository
 by Nature’s
 journal
 Scientific Data
  13. 13. MaX Centre of Excellence: AiiDA and Materials Cloud DISCOVER (CURATED DATA) & EXPLORE (RAW DATA) UUID links to jump to the
 provenance graph in the
 EXPLORE section DISCOVER EXPLORE Browse the full AiiDA
 provenance graph 
 (inputs, outputs, …) at any
 level
  14. 14. WORK: AiiDA Lab (submission) • Our cloud data generation platform and data analysis platform • Based on AiiDA + Jupyter + App Mode
  15. 15. WORK: AiiDA Lab (submission) • Our cloud data generation platform and data analysis platform • Based on AiiDA + Jupyter + App Mode Graph generated by the previous run
  16. 16. MaX Centre of Excellence: AiiDA and Materials Cloud WORK: AiiDA Lab
  17. 17. MaX Centre of Excellence: AiiDA and Materials Cloud Possible integration/collaboration points • Integration plans • AiiDA Lab • Deployment with kubernetes for autoscaling using EOSC services • Integration fo Authentication and Authorization with B2ACCESS and/or EGI Check-In • Registration of AiiDA Lab as a service on EOSC • Development and deployment of“turn-key”workflows for the materials science community as the“services” • Archive • Migration of the Materials Cloud Archive to Invenio v3 or EUDAT’s B2SHARE, still deployed on our premises • Integration in EUDAT’s B2FIND
  18. 18. MaX Centre of Excellence: AiiDA and Materials Cloud Possible integration/collaboration points • Integration plans • AiiDA Lab • Deployment with kubernetes for autoscaling using EOSC services • Integration fo Authentication and Authorization with B2ACCESS and/or EGI Check-In • Registration of AiiDA Lab as a service on EOSC • Development and deployment of“turn-key”workflows for the materials science community as the“services” • Archive • Migration of the Materials Cloud Archive to Invenio v3 or EUDAT’s B2SHARE, still deployed on our premises • Integration in EUDAT’s B2FIND EGICheck-in:
 scheduled W orkin
 progress Scheduled W orkin
 progress Scheduled B2ACCESS:
 done
  19. 19. MaX Centre of Excellence: AiiDA and Materials Cloud Acknowledgements: The AiiDA and Materials Cloud teams Giovanni
 Pizzi (EPFL) Boris
 Kozinsky (BOSCH) Martin
 Uhrin (EPFL) Spyros
 Zoupanos (EPFL) Nicola
 Marzari (EPFL) Snehal P.
 Kumbhar (EPFL) Leonid
 Kahle (EPFL) Sebastiaan
 P. Huber (EPFL) Marco Borelli (EPFL) Elsa Passaro (EPFL) Thomas Schulthess (ETHZ,CSCS) Leopold Talirz (EPFL) Joost VandeVondele (ETHZ,CSCS) Aliaksandr Yakutovich (EPFL) Contributors for the 25+ plugins Contributors to aiida_core and former team members — Valentin Bersier, Jocelyn Boullier, Jens Broeder, Andrea Cepellotti, Fernando Gargiulo, Dominik Gresch, Rico Häuselmann, Eric Hontz, Christoph Koch, Espen Flage-Larsen, Andrius Merkys, Nicolas Mounet, Tiziano Müller, Riccardo Sabatini, Ole Schütt, Phillippe Schwaller Berend
 Smit (EPFL) Casper W. Andersen (EPFL)
  20. 20. MaX Centre of Excellence: AiiDA and Materials Cloud Acknowledgements and funding H2020 Centre of Excellence“MaX” SNSF NCCR“MARVEL” Swissuniversities P-5“Materials Cloud” Discovery of new materials via simulations
 and dissemination of curated data Scaling towards exascale machines and 
 high-throughput efficiency Scaling the web platform, extending to more
 disciplines Moreover: H2020 Marketplace Providing data and simulation services in a EU 
 Marketplace platform for industry H2020 Intersect Develop AiiDA workflows to compute transport 
 properties of materials
  21. 21. MaX Centre of Excellence: AiiDA and Materials Cloud Conclusions • Open Science Platform for computational materials science: the MaX tool to converge HPC, HTC and HPDA • Guarantee reproducibility + FAIR sharing of data with their provenance • Define and run scientific workflows for materials and provide data analytics tools • Provide HPC workflows as a service + MaX can bring services to EOSC to streamline access to HPC
  22. 22. Contacts Materials Cloud: http://www.materialscloud.org - AiiDA Lab: http://aiidalab.materialscloud.org - Archive: http://archive.materialscloud.org @aiidateamhttps://www.facebook.com/aiidateam Website: http://www.aiida.net Docs: http://aiida-core.readthedocs.io Git repo: https://github.com/aiidateam/aiida_core/ Plugin registry: http://aiidateam.github.io/aiida-registry Quantum Mobile: http://www.materialscloud.org/work/ quantum-mobile Website: http://www.max-centre.eu @max_center2https://www.linkedin.com/company/max-center

×