Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Involvement in OpenHPC


Published on

Takeharu Kato Fujitsu
Nowadays, OpenHPC is gradually spreading as a software stack standard for HPC.
OpenHPC is one of the most promising software stack to achieve interoperability among HPC systems.
It is designed and developed to makes building HPC systems easier.
In this presentation, we explain the current status of OpenHPC and our involvement in OpenHPC to establish Arm HPC eco-system.

Published in: Technology
  • Be the first to comment

Involvement in OpenHPC

  1. 1. Copyright 2017 FUJITSU LIMITED Involvement in OpenHPC 12th December 2017 Takeharu KATO Takayuki OKAMOTO Software Development Div. Next Generation Technical Computing Unit FUJITSU LIMITED 0
  2. 2. Copyright 2017 FUJITSU LIMITED  Our activities in Linaro  What is OpenHPC?  Our contributions on OpenHPC  Current problem and our activities for solutions Agenda We prepare to build a standard software stack for HPC systems up. We explain our current results of the collaboration with Linaro. We also mention our current plan to improve OpenHPC with OpenHPC project. 1
  3. 3. Takeharu KATO Collaboration with Linaro Copyright 2017 FUJITSU LIMITED2
  4. 4. Fujitsu’s activities in Linaro Copyright 2017 FUJITSU LIMITED Contribution for ARM HPC eco-system  Purpose Improving ARM HPC systems with our experience and technologies around HPC systems.  Our current activities For making good use of HPC technologies in AArch64  AArch64 enablement of standard HPC software stack  Optimization with Scalable Vector Extensions(SVE)  Technical collaboration around SVE (e.g. simulator, compiler, and so on) 3
  5. 5. The necessity of the standard software stack for HPC Copyright 2017 FUJITSU LIMITED Open HPC is developed as a standard HPC software stack  To run HPC jobs, we need:  Work Load Manager/Resource Manager (Slurm/PBS)  Development tools(Compilers, profilers, debuggers)  Communication Libraries for parallel application(MPI)  Parallel scientific libraries (PETSc, FFTW...)  Serial scientific libraries (GNU Scientific library,...)  I/O libraries for scientific application (HDF5,NetCDF,...)  Necessity of the standard software stack We should gather software components by ourselves. But it is more preferable to share the best practice among HPC sites. A common software distribution for HPC systems is needed. 4
  6. 6. OpenHPC What is OpenHPC?  Software Constitution  All in one software suite for HPC e.g. WLM, Provisioning, MPI, Compilers, Scientific Libraries.  It makes software environment for both X86 and AArch64. Copyright 2017 FUJITSU LIMITED  Mission of OpenHPC  To provide a reference software collection with easy deployment  To share best practice, use of modern HPC methods and tools Base OS Development tools MPI Compilers I/O, Numerical/Scientific Libraries Provisioning Admin tools I/O Services(client) Work Load Managers Improving interoperability among HPC systems. 5
  7. 7. About OpenHPC Project  OpenHPC Project  A collaborative project in Linux Foundation  Consists of prominent academic organizations, research institutes, OEM vendors, ISV/OS vendors  OpenHPC development is governed under Technical Steering Committee(TSC).  ARM/Linaro(Fujitsu) are porting OpenHPC to AArch64. Copyright 2017 FUJITSU LIMITED OpenHPC Project standardize HPC soft stack for X86 and ARM. 6
  8. 8. Contribution from Fujitsu to OpenHPC  Improvement both functionality and performance of OpenHPC To make good use of the potential of ARM HPC, we contribute our experience and technologies on HPC systems(e.g. SVE). Copyright 2017 FUJITSU LIMITED ① Improving the variety of libraries ② Preparing develop/runtime environment for SVE. We collaborate with Linaro to upstream our work. It will help spread use of HPC on ARM. 7
  9. 9. Improving the variety of libraries Copyright 2017 FUJITSU LIMITED  Adding libraries to adapt to new trends of HPC. We added following libraries into OpenHPCv1.3.1:  Graph partitioning libraries  Math libraries to make good use of multicore architectures No Package Description 1 ParMetis Parallel Graph Partitioning and Fill- reducing Matrix Ordering 2 scotch graph and mesh/hypergraph partitioning 3 SLEPc the Scalable Library for Eigenvalue Problem Computations. 4 PLASMA Parallel Linear Algebra Software for Multicore Architectures [New packages in OpenHPC v1.3.1 from Linaro/Fujitsu ] 8
  10. 10. Enhancement with SVE  Our activities to spread use of SVE We are preparing the environment for SVE: ① Runtime environment Preparation of SVE runtime environment for application vendors (e.g. OSS Simulator for SVE application developers). ② Development environment Improvement optimization techniques of OSS compilers for HPC applications. Copyright 2017 FUJITSU LIMITED ① Sharing SVE simulation techniques and its codes. ② Improving compiler optimization for HPC applications. To prepare the environment before shipment of CPU Collaborations for making good use of the power of SVE 9
  11. 11. Takayuki OKAMOTO Collaboration with OpenHPC Project Copyright 2017 FUJITSU LIMITED10
  12. 12. Current Problems in OpenHPC  Several trivial bugs are found in each release of OpenHPC  e.g. A package naming convention violation only on aarch64 and SLES Copyright 2017 FUJITSU LIMITED Improvement of software configuration management is required both for ARM and for x86_64 underlying problem  The software configuration of OpenHPC became too complex to keep high quality  OpenHPC includes many software combinations •architectures(2) × OS(2) × installers(2) × schedulers(2) × compilers(2) × MPIs(2) × fabrics(2) = 128 patterns at least  Auto build system(OBS) and auto test system(Jenkins) are available but only a part of the patterns are tested. •13 patterns for x86_64 and 2 patterns for AArch64 are tested 11
  13. 13. Fujitsu's next step in OpenHPC  Restructuring the software configuration management  We have the experience to manage a commercial software configuration  These are important to keep a software quality •Make the definition of software configuration clear •Validate the software configuration at build time (lineup and dependency) •Build and test automatically to reduce the human work and human error  Collaboration with OpenHPC community We have just started to attend the TSC weekly teleconference last week. The second meeting will be held at this midnight.  To realize this big change  To contribute to keep high quality not only on ARM but also on x86_64 Copyright 2017 FUJITSU LIMITED12
  14. 14. Conclusion  Involvement in ARM HPC eco-system through Linaro  We contribute our experience and technologies on HPC systems to make good use of the potential of ARM HPC  Building a standard software stack for ARM HPC system by using OpenHPC  Adding some HPC libraries in OpenHPC  Enhancement with SVE  Expertise to optimize a compiler for HPC  Improving software configuration management Copyright 2017 FUJITSU LIMITED Cooperating with OSS communities to realize the high functionality and high performance HPC environment 13
  15. 15. Copyright 2017 FUJITSU LIMITED