Benefits of Multi-rail Cluster Architectures for GPU-based Nodes
•
3 likes•1,219 views
Craig Tierney from NVIDIA presented this deck at the MVAPICH User Group meeting. "As high performance computing moves toward GPU-accelerated architectures, single node application performance can be between 3x and 75x faster than the CPUs alone. Performance increases of this size will require increases in network bandwidth and message rate to prevent the network from becoming the bottleneck in scalability. In this talk, we will present results from NVLink enabled systems connected via quad-rail EDR Infiniband." Watch the video: https://wp.me/p3RLHQ-hkr Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Benefits of Multi-rail Cluster Architectures for GPU-based Nodes
Similar to Benefits of Multi-rail Cluster Architectures for GPU-based Nodes (20)
More from inside-BigData.com
More from inside-BigData.com (20)
Recently uploaded
Recently uploaded (20)
Benefits of Multi-rail Cluster Architectures for GPU-based Nodes
- 1. Craig Tierney, Solution Architect, NVIDIA August 15, 2017 TUNING MVAPICH2 FOR MULTI- RAIL INFININBAND AND NVLINK
- 2. 2 Agenda • Overview • Multi-rail • Intranode performance with NVLink
- 3. 3 GPUDIRECT FAMILY1 Technologies, enabling products !!! GPU pinned memory shared with other RDMA capable devices Avoids intermediate copies GPUDIRECT SHARED GPU- SYSMEM Accelerated GPU-GPU memory copies Inter-GPU direct load/store access GPUDIRECT P2P Direct GPU to 3rd party device transfers E.g. direct I/O, optimized inter-node communication GPUDIRECT RDMA2 Direct GPU to 3rd party device synchronizations E.g. optimized inter-node communication GPUDIRECT ASYNC [1] https://developer.nvidia.com/gpudirect [2] http://docs.nvidia.com/cuda/gpudirect-rdma
- 4. 4 GPUDIRECT FAMILY1 Technologies, enabling products !!! GPU pinned memory shared with other RDMA capable devices Avoids intermediate copies GPUDIRECT SHARED GPU- SYSMEM Accelerated GPU-GPU memory copies Inter-GPU direct load/store access GPUDIRECT P2P Direct GPU to 3rd party device transfers E.g. direct I/O, optimized inter-node communication GPUDIRECT RDMA2 Direct GPU to 3rd party device synchronizations E.g. optimized inter-node communication GPUDIRECT ASYNC [1] https://developer.nvidia.com/gpudirect [2] http://docs.nvidia.com/cuda/gpudirect-rdma Each platform has different performance characteristics, unique tuning required
- 5. 5 OVERVIEW • GPUs provide significantly more performance than CPUs for many HPC workloads Application Domain CPU Node Equivalence HOOMD-Blue Particle Dynamics 31 SPECFEM 3D Seismic Wave Propagation 78 VASP Quantum Chemistry 24 LSMS Materials Code 36 QUDA Lattice Quantum Chromo Dynamics 71 Number of nodes necessary to achieve same performance of 8-way P100 server. http://www.nvidia.com/object/application-performance-guide.html
- 6. 6 OVERVIEW • GPUs provide significantly more performance than CPUs for many HPC workloads Application Domain CPU Node Equivalence HOOMD-Blue Particle Dynamics 31 SPECFEM 3D Seismic Wave Propagation 78 VASP Quantum Chemistry 24 LSMS Materials Code 36 QUDA Lattice Quantum Chromo Dynamics 71 Number of nodes necessary to achieve same performance of 8-way P100 server. http://www.nvidia.com/object/application-performance-guide.html Takeaway: Network communications have to be highly efficient to applications to scale Single-rail networks may not be enough
- 7. 7 MULTI-RAIL APPLICATION PERFORMANCE 1.20 1.39 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1 2 3 4 RelativePerformance Number of Rails HPL Relative Performance (124 DGX Servers)
- 8. 8 TEST CONFIGURATION • Servers • 8 NVIDIA P100 GPUs /w NVLink • Intel Broadwell E5 v2697 • Quad-rail Mellanox EDR Infiniband • Switch • Mellanox SB7790, 36-port EDR switch • MPI – Mvpaich2-GDR 2.2 w/ GCC
- 9. 9 TEST CONFIGURATION • Servers • 8 NVIDIA P100 GPUs /w NVLink • Intel Broadwell E5 v2697 • Quad-rail Mellanox EDR Infiniband • Switch • Mellanox SB7790, 36-port EDR switch • MPI – Mvpaich2-GDR 2.2 w/ GCC Results presented here are intended to show general trends and not guarantee performance for any particular system or configuration.
- 10. 10 NVLINK • NVLink is an energy-efficient, high-bandwidth path between the GPUs and the CPU • NVLink Gen 1, Provides up to 160 GB/s • NVLink Gen 2, Provides up to 300 GB/s • IBM Power 8 (and future) systems connect GPUs directly to the CPU, removing the need for PCIe to communicate to the GPU
- 11. 11 SYSTEM ARCHITECTURE DIAGRAMS W/ NVLINK GPU1 GPU2 GPU0 GPU3 CPU0 CPU1 GPU0 GPU1 CPU0 CPU1 Ex: Supermicro 1028GQ-TRX IBM Power8 Minsky GPU0 GPU1
- 12. 12 SYSTEM ARCHITECTURE DIAGRAMS W/ NVLINK GPU0 GPU1 GPU2 GPU3 CPU0 GPU5 GPU4 GPU7 GPU6 CPU1 EX: Nvidia DGX Server
- 13. 13 Agenda • Overview • Multi-rail • Intranode performance with NVLink
- 15. 15 IB PERFORMANCE, OSU_BW Single-rail limit at 256k Caused by GDRCopy being enabled
- 18. 18 RANK LAYOUT ON MULTI-RAIL GPU SYSTEMS • While multi-rail performance is impressive, in practice there will be multiple ranks per node, most likely 1 rank per GPU • A better test is to measure bandwidth is to have one MPI rank per GPU • This is the OSU_MBW_MR test • The test has been modified to support device-to-device (DtD) as well as host-to- host (HtH) transfers
- 20. 20 IB PERFORMANCE, OSU_MBW_MR Dip in performance caused by lack of tuning
- 21. 21 IB PERFORMANCE, OSU_MBW_MR With affinity and tuning, DtD performance nearly matches HtH
- 22. 22 TUNABLE PARAMETERS MVAPICH2 Tunable Default Value Proposed Value Reason MV2_CUDA_IPC_THESHOLD 32768 262144 Improve DtD Intranode transfers at 32K and above. MV2_USE_GPUDIRECT_GDRCOPY_ LIMIT 8192 32768 Improve HtD Intranode transfers between 16K and 32K MV2_GPUDIRECT_LIMIT 8192 4194304* Improve Internode transfers MV2_USE_SMP_GDR 1 0 Offset performance degradation to Intranode transfers when setting MV2_GPDIRECT_LIMIT MV2_GPUDIRECT_RECEIVE_LIMIT 131072 4194304* Improve all Intranode transfers at 1M and above * May need to be larger depending on your largest transfer. Note, the search of this space was not exhaustive. More optimization can be done.
- 23. 23 Agenda • Overview • Multi-rail • Intranode performance with NVLink
- 24. 24 INTRANODE MPI PERFORMANCE • With the tunables above, what is the performance of intranode transfers? • Testing pairs: • GPUs 0 and 1 – NVLink connected, Same PCIe switch • GPUs 0 and 4 – NVLink connected, Different CPU socket • GPUs 0 and 6 – Not NVLink connected, Different CPU socket
- 27. 27 INTRANODE TRANSFERS - DEFAULTS
- 28. 28 INTRANODE TRANSFERS - TUNED
- 29. 29 INTRANODE TRANSFERS - DEFAULTS
- 30. 30 INTRANODE TRANSFERS - TUNED
- 31. 31 INTRANODE TRANSFERS - DEFAULTS
- 32. 32 INTRANODE TRANSFERS - TUNED
- 33. 33 INTRANODE TRANSFERS – BY GPU PAIR
- 34. 34 INTRANODE TRANSFERS – BY GPU PAIR
- 35. 35 INTRANODE TRANSFERS – BY GPU PAIR
- 36. 36 INTRANODE TRANSFERS – BY GPU PAIR Latency is similar for all cases as loopback IB is used
- 37. 37 SUMMARY • Mvapich2 is already very efficient with dual-rail and quad-rail configurations • Setting affinity properly for HCAs and GPUs is key to maximize total node BW • NVLink bandwidth provides great potential for improving scalability over PCIe • MPI point-to-point message passing can be greatly improved with the tunables provided in the Mvapich2 stack • More tuning will improve results • More tuning may discover parameters need to be scoped for intra- and inter- node transfers differently • Results should be rerun with mvpaich2-2.3a to see what changes have been made
- 38. Thank you!
- 39. 39 #!/bin/bash if [ $MV2_COMM_WORLD_LOCAL_RANK -eq 0 ]; then export LOCAL_RANK=$1 elif [ $MV2_COMM_WORLD_LOCAL_RANK -eq 1 ]; then export LOCAL_RANK=$2 else echo "THIS TOOL ONLY SUPPORTS 2 RANKS, EXITING" exit fi shift shift case ${LOCAL_RANK} in [0]) export MV2_IBA_HCA=mlx5_0 export CORELIST=0-4 ;; [1]) export MV2_IBA_HCA=mlx5_0 export CORELIST=5-9 ;; [2]) export MV2_IBA_HCA=mlx5_1 export CORELIST=10-14 ;; … [6]) export MV2_IBA_HCA=mlx5_3 export CORELIST=30-34 ;; [7]) export MV2_IBA_HCA=mlx5_3 export CORELIST=35-39 ;; esac export MV2_ENABLE_AFFINITY=0 numactl --physcpubind=$CORELIST $*