Successfully reported this slideshow.
Your SlideShare is downloading. ×

Monitoring and Operational Data Analytics from a User Perspective at First EuroCC HPC Vega Supercomputer and Nation-wide in Slovenia

Ad

Introduction Challenges Initiatives Tools Conclusion References
Monitoring and Operational Data Analytics
from a User Pers...

Ad

Introduction Challenges Initiatives Tools Conclusion References
Introduction: Aims of this Talk at MODA21 Workshop
This co...

Ad

Introduction Challenges Initiatives Tools Conclusion References
Introduction: Outline of the Talk
I First (5 minutes): Cha...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Check these out next

1 of 28 Ad
1 of 28 Ad

Monitoring and Operational Data Analytics from a User Perspective at First EuroCC HPC Vega Supercomputer and Nation-wide in Slovenia

https://moda21.sciencesconf.org/resource/page/id/4
https://www.isc-hpc.com/
https://twitter.com/aleszamuda/status/1410717857864953860?s=20

https://moda21.sciencesconf.org/resource/page/id/4
https://www.isc-hpc.com/
https://twitter.com/aleszamuda/status/1410717857864953860?s=20

Advertisement
Advertisement

More Related Content

Advertisement

Monitoring and Operational Data Analytics from a User Perspective at First EuroCC HPC Vega Supercomputer and Nation-wide in Slovenia

  1. 1. Introduction Challenges Initiatives Tools Conclusion References Monitoring and Operational Data Analytics from a User Perspective at First EuroCC HPC Vega Supercomputer and Nation-wide in Slovenia an Invited talk, 16:20 - 16:40 CET at MODA 21: Monitoring and Operational Data Analytics (workshop held at ISC 2021 on July 2, 2021) ISC High Performance 2021 Digital June 24 - July 2, 2021 Aleš Zamuda ales.zamuda@um.si Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 1/ 28
  2. 2. Introduction Challenges Initiatives Tools Conclusion References Introduction: Aims of this Talk at MODA21 Workshop This contribution focuses on I collecting, analyzing, and visualizing I rich system and application data, and I my opinion on how one can make sense of the data for actionable insights. I Explained through examples: from a HPC User Perspective. Real examples: science and HPC Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 2/ 28
  3. 3. Introduction Challenges Initiatives Tools Conclusion References Introduction: Outline of the Talk I First (5 minutes): Challenges faced challenges, leading to the needs to apply HPC architectures for benchmarking state-of-the-art topics in I Second (2 minutes): Initiatives introduction of available HPC initiatives (nearby and wider). I Third (7 minutes): MODA MODA tools leveraged in the example use of HPC for text summarization. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 3/ 28
  4. 4. Introduction Challenges Initiatives Tools Conclusion References Challenges (First part) Faced 5 types of challenges, leading to the needs to apply HPC architectures for benchmarking state-of-the-art topics in 1. forest ecosystem modeling, simulation, and visualization, 2. underwater robotic mission planning, 3. energy production scheduling for hydro-thermal power plants, 4. understanding evolutionary algorithms, and 5. text summarization. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 4/ 28
  5. 5. Introduction Challenges Initiatives Tools Conclusion References Challenges 1: Forest Ecosystem Modeling, Simulation, and Visualization I HPC need to process spatial data and add procedural content. Videos: https://www.youtube.com/watch?list=PL7pmTW8neV7tZf2qx1wV5zbD74sUyHL3B&v=V9YJgYO_sIA Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 5/ 28
  6. 6. Introduction Challenges Initiatives Tools Conclusion References Challenges 2: Underwater Robotic Mission Planning I Computational Fluid Dynamics (CFD) spatio-temporal model of the ocean currents for autonomous vehicle navigation path planning. I Constrained Differential Evolution Optimization for Underwater Glider Path Planning in Sub-mesoscale Eddy Sampling. I Corridor-constrained optimization: eddy border region sampling — new challenge for UGPP & DE. I Feasible path area is constrained — trajectory in corridor around the border of an ocean eddy. The objective of the glider here is to sample the oceanographic variables more efficiently, while keeping a bounded trajectory. HPC: develop new methods and evaluate them. Video: https://www.youtube.com/watch?v=4kCsXAehAmU Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 6/ 28
  7. 7. Introduction Challenges Initiatives Tools Conclusion References Challenges 3: Energy Production Scheduling for Hydro-thermal Power Plants A. Glotić, A. Zamuda. Short-term combined economic and emission hydrothermal optimization by surrogate differential evolution. Applied Energy, 1 March 2015, vol. 141, pp. 42-56. DOI: 10.1016/j.apenergy.2014.12.020 Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 7/ 28
  8. 8. Introduction Challenges Initiatives Tools Conclusion References Challenges 4: Understanding Evolutionary Algorithms I Evolutionary algorithms benchmarking to understand computational intelligence of these algorithms (→ storage requirement!), I aim: Machine Learning to design an optimization algorithm (learning to learn). I Example CI Algorithm Mechanism Design: Control Parameters Self-Adaptation (in DE). Video: https://www.youtube.com/watch?v=R244LZpZSG0 Application stacks for real code: inspired by previous computational optimization competitions in continuous settings that used test functions for optimization application domains: I single-objective: CEC 2005, 2013, 2014, 2015 I constrained: CEC 2006, CEC 2007, CEC 2010 I multi-modal: CEC 2010, SWEVO 2016 I black-box (target value): BBOB 2009, COCO 2016 I noisy optimization: BBOB 2009 I large-scale: CEC 2008, CEC 2010 I dynamic: CEC 2009, CEC 2014 I real-world: CEC 2011 I computationally expensive: CEC 2013, CEC 2015 I learning-based: CEC 2015 I 100-digit (50% targets): 2019 joined CEC, SEMCCO, GECCO I multi-objective: CEC 2002, CEC 2007, CEC 2009, CEC 2014 I bi-objective: CEC 2008 I many objective: CEC 2018 Tuning/ranking/hyperheuristics use. → DEs as usual winner algorithms. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 8/ 28
  9. 9. Introduction Challenges Initiatives Tools Conclusion References Challenges 5: Text Summarization For NLP, part of ”Big Data”. Terms across sentences are determined using a semantic analysis using both: I coreference resolution (using WordNet) and I a Concept Matrix (from Freeling). INPUT CORPUS NATURAL LANGUAGE PROCESSING ANALYSIS CONCEPTS DISTRIBUTION PER SENTENCES MATRIX OF CONCEPTS PROCESSING CORPUS PREPROCESSING PHASE OPTIMIZATION TASKS EXECUTION PHASE ASSEMBLE TASK DESCRIPTION SUBMIT TASKS TO PARALLEL EXECUTION OPTIMIZER + TASK DATA ROUGE EVALUATION The detailed new method called CaBiSDETS is developed in the HPC approach comprising of: I a version of evolutionary algorithm (Differential Evolution, DE), I self-adaptation, binarization, constraint adjusting, and some more pre-computation, I optimizing the inputs to define the summarization optimization model. Aleš Zamuda, Elena Lloret. Optimizing Data-Driven Models for Summarization as Parallel Tasks. Journal of Computational Science, 2020, vol. 42, pp. 101101. DOI: 10.1016/j.jocs.2020.101101 Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 9/ 28
  10. 10. Introduction Challenges Initiatives Tools Conclusion References Initiatives (Second part) Timeline (as member) of recent impactful HPC initiatives including Slovenia: I SLING: Slovenian national supercomputing network, 2010-05-03–, I SIHPC: Slovenian High-Performance Computing Centre, 2016-03-04– I ImAppNIO: Improving Applicability of Nature-Inspired Optimisation by Joining Theory and Practice, 2016-03-09–2020-10-31 I cHiPSet: High-Performance Modelling and Simulation for Big Data Applications, 2015-04-08–2019-04-07, I HPC RIVR: UPGRADING OF NATIONAL RESEARCH INFRASTRUCTURES, Investment Program, 2018-03-01–2020-09-15, I TFoB: IEEE CIS Task Force on Benchmarking, January 2020–, I EuroCC: National Competence Centres in the framework of EuroHPC, 2020-09-01–(2022-08-31), I DAPHNE: Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC, and Machine Learning, 2020-12-01–(2024-11-24). Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 10/ 28
  11. 11. Introduction Challenges Initiatives Tools Conclusion References Initiatives: SLING, SIHPC, HPC RIVR, EuroCC I There is a federated and orchestrated aim towards HPC infrastructure in Slovenia, especially through: I SLING: Slovenian national supercomputing network → has fedarated the initiative push towards orchestration of HPC resources across the country. I SIHPC: Slovenian High-Performance Computing Centre → has orchestrated the first EU funds application towards HPC Teaming in the country (and Participation of Slovenia in PRACE 2). I HPC RIVR: UPGRADING OF NATIONAL RESEARCH INFRASTRUCTURES, Investment Program → has provided an investment in experimental HPC infrastructure. I EuroCC: National Competence Centres in the framework of EuroHPC → has secured a National Competence Centre (EuroHPC). Vega supercomputer online Consortium Slovenian High-Performance Computing Centre Aškerčeva ulica 6 SI-1000 Ljubljana Slovenia Ljubljana, 22. 2. 2017 prof. dr. Anwar Osseyran PRACE Council Chair Subject: Participation of Slovenia in PRACE 2 Dear professor Osseyran, In March 2016 a consortium Slovenski superračunalniški center (Slovenian High-Performance Computing Centre - SIHPC) was established with a founding act (Attachment 1) where article 6 claims that the consortium will join PRACE and that University of Ljubljana, Faculty of mechanical engineering (ULFME) will represent the consortium in the PRACE. The legal representative of ULFME in the consortium and therefore, also in the PRACE (i.e., the delegate in PRACE Council with full authorization) is prof. dr. Jožef Duhovnik, as follows from appointment of the dean of ULFME (Attachment 2). The consortium has also agreed that it joins PRACE 2 optional programme as contributing GP. With best regards, assist. prof. dr. Aleš Zamuda, vice-president of SIHPC Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 11/ 28
  12. 12. Introduction Challenges Initiatives Tools Conclusion References Initiatives: ImAppNIO, cHiPSet, TFoB, DAPHNE Aim towards software to run HPC and improve capabilities: I ImAppNIO: Improving Applicability of Nature-Inspired Optimisation by Joining Theory and Practice, → improve capabilities through benchmarking (to understand (and to learn to learn)) CI algorithms I cHiPSet: High-Performance Modelling and Simulation for Big Data Applications, → include HPC in Modelling and Simulation (of the process to be learned) I TFoB: IEEE CIS Task Force on Benchmarking, → includes CI benchmarking opportunities, where HPC would enable new capabilities. I DAPHNE: Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC, and Machine Learning. → to define and build an open and extensible system infrastructure for integrated data analysis pipelines, including data management and processing, high-performance computing (HPC), and machine learning (ML) training and scoring https://daphne-eu.github.io/ Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 12/ 28
  13. 13. Introduction Challenges Initiatives Tools Conclusion References Tools (Third part) MODA (Monitoring and Operational Data Analytics) tools for I collecting, analyzing, and visualizing I rich system and application data, and I my opinion on how one can make sense of the data for actionable insights. I Explained through previous examples: from a HPC User Perspective. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 13/ 28
  14. 14. Introduction Challenges Initiatives Tools Conclusion References MODA Actionable Insights, Explained From a HPC User Perspective, Through the Example of Summarization Most interesting findings of summarization on HPC example are I the fitness of the NLP model keeps increasing with prolonging the dedicated HPC resources (see below) and that I the fitness improvement correlates with ROUGE evaluation in the benchmark, i.e. better summaries. -0.05 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 1 10 100 1000 10000 ROUGE-1R ROUGE-2R ROUGE-LR ROUGE-SU4R Fitness (scaled) Hence, the use of HPC significantly contributes to capability of this NLP challenge. However, the MODA insight also provided the useful task running times and resource usage. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 14/ 28
  15. 15. Introduction Challenges Initiatives Tools Conclusion References Running the Tasks on HPC: ARC Job Preparation Parallel summarization tasks on grid through ARC. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 15/ 28
  16. 16. Introduction Challenges Initiatives Tools Conclusion References Running the Tasks on HPC: ARC Job Submission, Results Retrieval & Merging [JoCS2020] Through an HPC approach and by parallelization of tasks, a data-driven summarization model optimization yields improved benchmark metric results (drawn using gnuplot merge). MODA is needed to run again and improve upon, to forecast how to set required task running time and resources (predicting system response). Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 16/ 28
  17. 17. Introduction Challenges Initiatives Tools Conclusion References Monitoring and Operational Data Analytics I Monitor used (jobs, CPU/wall time, etc.): Smirnova, Oxana. The Grid Monitor. Usage manual, Tech. Rep. NORDUGRID-MANUAL-5, The NorduGrid Collaboration, 2003. http://www.nordugrid.org/documents/ http://www.nordugrid.org/manuals.html http://www.nordugrid.org/documents/monitor.pdf I Deployed at: www.nordugrid.org/monitor/ I NorduGrid Grid Monitor Sampled: 2021-06-28 at 17-57-08 I Nation-wide in Slovenia: https://www.sling.si/gridmonitor/loadmon.php http://www.nordugrid.org/monitor/index.php? display=vo=Slovenia Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 17/ 28
  18. 18. Introduction Challenges Initiatives Tools Conclusion References MODA Example From: ARC at Jost Example experiments from DOI: 10.1016/j.swevo.2015.10.007 (SPSRDEMMS) – job YYULDmGOXpmnmmR0Xox1SiGmABFKDmABFKDmrtMKDmABFKDm66faPo. Sample ARC file gridlog/diag (2–3 day Wall Times). runtimeenvironments=APPS/ARNES/MPI−1.6−R; CPUUsage=99% MaxResidentMemory=5824kB AverageUnsharedMemory=0kB AverageUnsharedStack=0kB AverageSharedMemory=0kB PageSize =4096B MajorPageFaults=4 MinorPageFaults =1213758 Swaps=0 ForcedSwitches =36371494 WaitSwitches =170435 I n p u t s =45608 Outputs =477168 SocketReceived=0 SocketSent=0 S i g n a l s=0 nodename=wn003 . arnes . s i WallTime=148332 s P r o c e s s o r s =16 UserTime =147921.14 s KernelTime =2.54 s AverageTotalMemory=0kB AverageResidentMemory=0kB LRMSStartTime =20150906104626Z LRMSEndTime=20150908035838Z e x i t c o d e=0 Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 18/ 28
  19. 19. Introduction Challenges Initiatives Tools Conclusion References EuroCC HPC: Vega (TOP500 #106, HPCG #56 — June 2021) I Researchers can apply to EuroHPC JU calls for access. I Regular calls begin this fall (Benchmark & Development). I https://prace-ri.eu/benchmark-and-development-access-information-for-applicants/ I 60% capacities for national share (70% OA, 20% commercial, 10% host (community, urgent priority of national importance, maintenance)) + 35% EuroHPC JU share (approved applications) I Has a SLURM dev partition for SSH login (SLURM partitions w/ CPUs: login[0001-0004]=128; login[0005-0008]=64; cn[0001-384,0577-0960]=256; cn[0385-0576]=256; gn[01-60]=256). Listing 1: Setting up at Vega — slurm dev partition access (login). 1 [ a l e s . zamuda@vglogin0007 ˜] $ s i n g u l a r i t y p u l l qmake . s i f docker :// ak352/qmake−opencv 2 [ a l e s . zamuda@vglogin0007 ˜] $ s i n g u l a r i t y run qmake . s i f bash 3 cd sum ; qmake ; make c l e a n ; make 4 5 [ a l e s . zamuda@vglogin0007 ˜] $ cat runme . sh 6 #!/ bin / bash 7 cd sum && time mpirun 8 − −mca b t l o p e n i b w a r n n o d e v i c e p a r a m s f o u n d 0 9 ./ summarizer 10 − −useBinaryDEMPI − −i n p u t f i l e mRNA−1273−t x t 11 − −withoutStatementMarkersInput 12 − −printPreprocessProgress calcInverseTermFrequencyndTermWeights 13 − −p r i n t O p t i m i z a t i o n B e s t I n G e n e r a t i o n 14 − −summarylength 600 − −NP 200 15 − − GMAX 400 16 > summarizer . out . $SLURM PROCID 17 2> summarizer . e r r . $SLURM PROCID -0.65 -0.6 -0.55 -0.5 -0.45 -0.4 -0.35 -0.3 -0.25 -0.2 -0.15 -0.1 1 10 100 Evaluation A protective serum NAb titer in the 300 to 1,000 range can be in- ferred. Since the 1990’s, significant progress has also been made in developing flexible, amplifiable, scalable, inexpensive, and cold-chain free RNA vaccines, such as synthetic mRNA molecules encoding only the antigen of interest and self-amplifying RNA (sa-RNA) (264). Us- ing the same technology, researchers from Oxford University devel- oped ChAdOx1, which will eventually produce spike proteins within the human body, leading to immune system activation. Inovio Phar- maceuticals, a US-based biotechnology firm, has developed INO-4800, a DNA-based vaccine, utilizing a relatively newer vaccine technique. Shenzhen Geno-Immune Medical Institute has generated dendritic cells (DCs) (LV-SMENP-DC in Phase 1, NCT042276896) and artificial APCs (aAPCs) modified with a lentiviral vector expressing a synthetic minigene based on parts of selected viral proteins (pathogen-specific aAPC, Phase I, NCT04299724. Additionally, timing for optimal ef- fect, commercial availability, and production scalability of the vaccine are major issues during a pandemic133. The secondary objective is to evaluate the immunogenicity measured by ELISA to the SARS-CoV-2 Spike protein following a 2-dose vaccination schedule of mRNA-1273 at Day 57 [165]. Additionally, it is also being evaluated in 2 Phase 3 clinical trials (NCT04456595 and 669/UN6. Two days after chal- lenge, only one of eight animals in each of the vaccine dose groups had detectable subgenomic RNA in BAL fluid, as compared with eight of eight animals in the control group (Figure 3A). Out of these, 73 are in preclinical stages, and a few of them have moved to clinical trials like mRNA-1273, Ad5-nCoV, INO-4800, pathogen-specific aAPC, etc . Evaluation of the mRNA-1273 vaccine against SARS-CoV-2 in nonhu- man primat. Early phase I/II safety and efficacy studies showed that BNT162 indicated transient mild to moderate local reactions and sys- temic events that were dose dependent and led to RBD-binding IgG concentrations and SARS-CoV-2 neutralizing titers in sera that in- creased with the dose level and after a second dose [102]. Information about specific SARS- CoV-2 antigen(s) employed for vaccine develop- ment is publicly limited. The results of the trial are promising as the mRNA-1273 vaccine elicited an anti-SARS-CoV-2 immune response in all participants, with no trial-limiting safety concerns [75. Overall, these data suggest the potential to develop DNA and mRNA vaccines that are easier to design and can quickly proceed into clinical trials, which will be helpful for pandemic states such as the one caused by COVID-19 [77. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 19/ 28
  20. 20. Introduction Challenges Initiatives Tools Conclusion References MODA at First EuroCC HPC Vega Supercomputer Listing 2: Runnig at Vega & MODA. 1 ===================================================================== GMAX=200 ===== 2 [ a l e s . zamuda@vglogin0002 ˜] $ srun − −cpu−bind=c o r e s − −nodes=1 − −ntasks−per−node=101 3 − −cpus−per−task=2 − −mem=180G s i n g u l a r i t y run qmake . s i f bash 4 srun : job 4531374 queued and w a i t i n g f o r r e s o u r c e s 5 srun : job 4531374 has been a l l o c a t e d r e s o u r c e s 6 [ ”$SLURM PROCID” = 0 ] && ./ runme . sh 7 r e a l 5m22.475 s 8 u s e r 484m42.262 s 9 s y s 1m38.304 s 10 ===================================================================== NODES=51 ===== 11 [ a l e s . zamuda@vglogin0002 ˜] $ srun − −cpu−bind=c o r e s − −nodes=1 − −ntasks−per−node=51 12 − −cpus−per−task=2 − −mem=180G s i n g u l a r i t y run qmake . s i f bash 13 srun : job 4531746 queued and w a i t i n g f o r r e s o u r c e s 14 srun : job 4531746 has been a l l o c a t e d r e s o u r c e s 15 [ ”$SLURM PROCID” = 0 ] && ./ runme . sh 16 r e a l 13m57.851 s 17 u s e r 431m25.833 s 18 s y s 0m29.272 s 19 ===================================================================== GMAX=400 ===== 20 [ a l e s . zamuda@vglogin0002 ˜] $ srun − −cpu−bind=c o r e s − −nodes=1 − −ntasks−per−node=101 21 − −cpus−per−task=2 − −mem=180G s i n g u l a r i t y run qmake . s i f bash 22 srun : job 4532697 queued and w a i t i n g f o r r e s o u r c e s 23 srun : job 4532697 has been a l l o c a t e d r e s o u r c e s 24 [ ”$SLURM PROCID” = 0 ] && ./ runme . sh 25 r e a l 6m14.687 s 26 u s e r 590m45.641 s 27 s y s 1m40.930 s Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 20/ 28
  21. 21. Introduction Challenges Initiatives Tools Conclusion References More Output: Shell is Flexible! Listing 3: Example accounting tool at Vega: sacct. [ a l e s . zamuda@vglogin0002 ˜] $ s a c c t 4531374. ext+ e x t e r n vega−u s e r s 202 COMPLETED 0:0 4531746. ext+ e x t e r n vega−u s e r s 102 COMPLETED 0:0 4532697. ext+ e x t e r n vega−u s e r s 202 COMPLETED 0:0 [ a l e s . zamuda@vglogin0002 ˜] $ s a c c t −j 4531374 −j 4531746 −j 4532697 −o MaxRSS , MaxVMSize , AvePages MaxRSS MaxVMSize AvePages − − − − − − − − − − − − − − − − − − − − − − − − − − − − − − 0 217052K 0 26403828K 1264384K 22 0 217052K 0 13325268K 1264380K 0 0 217052K 0 26404356K 1264384K 30 Future MODA testings: I testing the web interface for job analysis (as available from HPC RIVR); I profiling MPI inter-node communication; I use profilers and monitoring tools available — in the context of heterogeneous setups, like e.g. I TAU Performance System — http://www.cs.uoregon.edu/research/tau/home.php, I LIKWID Performance Tools — https://hpc.fau.de/research/tools/likwid/. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 21/ 28
  22. 22. Introduction Challenges Initiatives Tools Conclusion References More HPC User Perspective Nation-wide in Slovenia More: at University of Maribor, Bologna study courses for teaching (training) of Computer Science at cycles — click URL: I level 1 (BSc) I year 1: Programming I – e.g. C++ syntax I year 2: Computer Architectures – e.g. assembly, microcode, ILP I year 3: Parallell and Distributed Computing – e.g. OpenMP, MPI, CUDA I level 2 (MSc) I year 1: Cloud Computing Deployment and Management – e.g. arc, slurm, Hadoop, containers (docker, singularity) through virtualization I level 3 (PhD) I EU and other national projects research: HPC RIVR, EuroCC, DAPHNE, ... – e.g. scaling new systems of CI & Operational Research of ... over HPC I IEEE Computational Intelligence Task Force on Benchmarking I Scientific Journals (e.g. SWEVO, TEVC, JoCS, ASOC, INS) These contribute towards Sustainable Development of HPC. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 22/ 28
  23. 23. Introduction Challenges Initiatives Tools Conclusion References Conclusion Summary: HPC challenges, initiatives, and MODA tools. Thanks! Questions? -0.05 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 1 10 100 1000 10000 ROUGE-1R ROUGE-2R ROUGE-LR ROUGE-SU4R Fitness (scaled) Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 23/ 28
  24. 24. Introduction Challenges Initiatives Tools Conclusion References Biography and References: Organizations I Associate Professor at University of Maribor, Slovenia I Continuous research programme funded by Slovenian Research Agency, P2-0041: Computer Systems, Methodologies, and Intelligent Services I Associate Editor: Swarm and Evolutionary Computation (SWEVO) I IEEE (Institute of Electrical and Electronics Engineers) senior I IEEE Computational Intelligence Society (CIS), senior member I IEEE CIS Task Force on Benchmarking, chair I IEEE CIS, Slovenia Section Chapter (CH08873), chair I IEEE Slovenia Section, vice chair I IEEE Young Professionals Slovenia, past chair I ACM SIGEVO (Special Interest Group on Genetic and Evolutionary Computation); EurAI; SLAIS I Co-operation in Science and Techology (COST) Association Management Committee, member: I CA COST Action CA15140: Improving Applicability of Nature-Inspired Optimisation by Joining Theory and Practice (ImAppNIO), WG3 VC I ICT COST Action IC1406 High-Performance Modelling and Simulation for Big Data Applications (cHiPSet); SI-HPC; HPC-RIVR user I EU H2020 Research and Innovation project, holder for UM part: Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC, and Machine Learning (DAPHNE), https://cordis.europa.eu/project/id/957407 Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 24/ 28
  25. 25. Introduction Challenges Initiatives Tools Conclusion References Biography and References: Top Publications I Aleš Zamuda, Elena Lloret. Optimizing Data-Driven Models for Summarization as Parallel Tasks. Journal of Computational Science, 2020, vol. 42, pp. 101101. DOI 10.1016/j.jocs.2020.101101. I A. Zamuda, J. D. Hernández Sosa. Success history applied to expert system for underwater glider path planning using differential evolution. Expert Systems with Applications, 2019, vol. 119, pp. 155-170. DOI 10.1016/j.eswa.2018.10.048 I A. Viktorin, R. Senkerik, M. Pluhacek, T. Kadavy, A. Zamuda. Distance Based Parameter Adaptation for Success-History based Differential Evolution. Swarm and Evolutionary Computation, 2019, vol. 50, pp. 100462. DOI 10.1016/j.swevo.2018.10.013. I A. Zamuda, J. Brest. Self-adaptive control parameters’ randomization frequency and propagations in differential evolution. Swarm and Evolutionary Computation, 2015, vol. 25C, pp. 72-99. DOI 10.1016/j.swevo.2015.10.007. I A. Zamuda, J. D. Hernández Sosa, L. Adler. Constrained Differential Evolution Optimization for Underwater Glider Path Planning in Sub-mesoscale Eddy Sampling. Applied Soft Computing, 2016, vol. 42, pp. 93-118. DOI 10.1016/j.asoc.2016.01.038. I A. Zamuda, J. D. Hernández Sosa. Differential Evolution and Underwater Glider Path Planning Applied to the Short-Term Opportunistic Sampling of Dynamic Mesoscale Ocean Structures. Applied Soft Computing, vol. 24, November 2014, pp. 95-108. DOI 10.1016/j.asoc.2014.06.048. I A. Zamuda, J. Brest. Vectorized Procedural Models for Animated Trees Reconstruction using Differential Evolution. Information Sciences, vol. 278, pp. 1-21, 2014. DOI 10.1016/j.ins.2014.04.037. I A. Zamuda, J. Brest. Environmental Framework to Visualize Emergent Artificial Forest Ecosystems. Information Sciences, vol. 220, pp. 522-540, 2013. DOI 10.1016/j.ins.2012.07.031. I A. Glotić, A. Zamuda. Short-term combined economic and emission hydrothermal optimization by surrogate differential evolution. Applied Energy, 1 March 2015, vol. 141, pp. 42-56. DOI 10.1016/j.apenergy.2014.12.020. I H. Hamann, Y. Khaluf, J. Botev, M. Divband Soorati, E. Ferrante, O. Kosak, J.-M. Montanier, S. Mostaghim, R. Redpath, J. Timmis, F. Veenstra, M. Wahby and A. Zamuda. Hybrid Societies: Challenges and Perspectives in the Design of Collective Behavior in Self-organizing Systems. Frontiers in Robotics and AI, 2016, vol. 3, no. 14. DOI 10.3389/frobt.2016.00014. I J. Šilc, A. Zamuda. Special Issue on ”Bioinspired Optimization” (guest editors). Informatica - An International Journal of Computing and Informatics, 2015, vol. 39, no. 2, pp. 1-122. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 25/ 28
  26. 26. Introduction Challenges Initiatives Tools Conclusion References Biography and References: Bound Specific to HPC PROJECTS: I DAPHNE: Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC, and Machine Learning I ICT COST Action IC1406 High-Performance Modelling and Simulation for Big Data Applications I SLING: Slovenian national supercomputing network I SI-HPC: Slovenian corsortium for High-Performance Computing I UM HPC-RIVR: Supercomputer at UM, https://www.hpc-rivr.si/ I SmartVillages: Smart digital transformation of villages in the Alpine Space I Interreg Alpine Space, https://www.alpine-space.eu/projects/smartvillages/en/home I Interactive multimedia digital signage (PKP, Adin DS) EDITOR: I SWEVO (Top Journal), Associate Editor I Mathematics-MDPI, Special Issue: Evolutionary Algorithms in Engineering Design Optimization I Journal of advanced engineering and computation (member of editorial board since 2019). Viet Nam: Ton Duc Thang University, 2017-. ISSN 2588-123X. I Cloud Computing and Data Science (Associate Editor, since 2019). Universal Wiser Publisher Pte.Ltd. I D. Gleich, P. Planinšič, A. Zamuda. 2018 25th International Conference on Systems, Signals and Image Processing (IWSSIP). IEEE Xplore, Maribor, 20-22 June 2018. I General Chair: 7-th Joint International Conferences on Swarm, Evolutionary and Memetic Computing Conference (SEMCCO 2019) & Fuzzy And Neural Computing Conference (FANCCO 2019), Maribor, Slovenia, EU, 10-12 July 2019; co-editors: Aleš Zamuda, Swagatam Das, Ponnuthurai Nagaratnam Suganthan, Bijaya Ketan Panigrahi. Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 26/ 28
  27. 27. Introduction Challenges Initiatives Tools Conclusion References Biography and References: More on HPC RESEARCH PUBLICATIONS: I Aleš Zamuda, Elena Lloret. Optimizing Data-Driven Models for Summarization as Parallel Tasks. Journal of Computational Science, 2020, vol. 42, pp. 101101. DOI 10.1016/j.jocs.2020.101101. I Aleš Zamuda, Vincenzo Crescimanna, Juan C. Burguillo, Joana Matos Dias, Katarzyna Wegrzyn-Wolska, Imen Rached, Horacio González-Vélez, Roman Senkerik, Claudia Pop, Tudor Cioara, Ioan Salomie, Andrea Bracciali. Forecasting Cryptocurrency Value by Sentiment Analysis: An HPC-Oriented Survey of the State-of-the-Art in the Cloud Era. Kolodziej J., González-Vélez H. (eds) High-Performance Modelling and Simulation for Big Data Applications. Lecture Notes in Computer Science, vol 11400, 2019, pp. 325-349. DOI 10.1007/978-3-030-16272-6 12. I Nenad Karolija, Aleš Zamuda. On cloud-supported web-based integrated development environment for programming dataflow architectures. MILUTINOVIĆ, Veljko (ur.), KOTLAR, Milos (ur.). Exploring the DataFlow supercomputing paradigm: example algorithms for selected applications, (Computer communications and networks (Internet), ISSN 2197-8433), 2019, pp. 41-51. DOI 10.1007/978-3-030-13803-5 2. I Simone Spolaor, Marco Gribaudo, Mauro Iacono, Tomas Kadavy, Zuzana Komı́nková Oplatková, Giancarlo Mauri, Sabri Pllana, Roman Senkerik, Natalija Stojanovic, Esko Turunen, Adam Viktorin, Salvatore Vitabile, Aleš Zamuda, Marco S. Nobile. Towards Human Cell Simulation. Kolodziej J., González-Vélez H. (eds) High-Performance Modelling and Simulation for Big Data Applications. Lecture Notes in Computer Science, vol 11400, 2019, pp. 221-249. DOI 10.1007/978-3-030-16272-6 8. I A. Zamuda, J. D. Hernandez Sosa, L. Adler. Improving Constrained Glider Trajectories for Ocean Eddy Border Sampling within Extended Mission Planning Time. IEEE Congress on Evolutionary Computation (CEC) 2016, 2016, pp. 1727-1734. I A. Zamuda. Function evaluations upto 1e+12 and large population sizes assessed in distance-based success history differential evolution for 100-digit challenge and numerical optimization scenarios (DISHchain 1e+12): a competition entry for ”100-digit challenge, and four other numerical optimization competitions” at the genetic and evolutionary computation conference (GECCO) 2019. Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO 2019), 2019, pp. 11-12. I ... several more experiments for papers run using HPCs. I ... also, pedagogic materials in Slovenian and English — see Conclusion . Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 27/ 28
  28. 28. Introduction Challenges Initiatives Tools Conclusion References Promo materials: Calls for Papers, Informational Websites CS FERI WWW CIS TFoB CFPs WWW LinkedIn Twitter Aleš Zamuda 7@aleszamuda MODA from a User Perspective at 1st EuroCC HPC Vega Supercomputer & SI-wide 28/ 28

×