Download as PDF, PPTX
Uploaded byJeff Larkin
Optimizing GPU to GPU Communication on Cray XK7
The document discusses the optimization of GPU-to-GPU communication on Cray XK7 systems, highlighting challenges such as higher latency and lower bandwidth when utilizing traditional methods. It emphasizes the importance of GPU-aware MPI libraries that simplify the programming model and enhance performance through techniques like RDMA and pipelining. Ultimately, the paper argues that optimizing communication is crucial as computation scales, and developing effective communication strategies can exploit natural concurrency in certain patterns.
More Related Content
Similar to Optimizing GPU to GPU Communication on Cray XK7
![[Harvard CS264] 07 - GPU Cluster Programming (MPI & ZeroMQ)](https://cdn.slidesharecdn.com/ss_thumbnails/cs264201107-mpi0mq-110308182928-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Optimizing GPU to GPU Communication on Cray XK7
- 1. Optimizing GPU toGPU Communication on Cray XK7 Jeff Larkin
- 2. What Amdahl saysabout GPU communication If you make your GPU computation infinitely fast, performance will be bound by your communication. GPU-2-GPU communication has Higher latency (additional hop over PCIe) Lower bandwidth (limited by lowest bandwidth link) G2G communication cannot be an afterthought when running at scale.
- 3. CPU0 MPI_Send () GPU0 CPU1 MPI_Recv() GPU1 How doGPUs communicate? MPI But what really happens here?
- 4.
- 5. Unified Virtual Addressing NoUVA: Multiple Memory Spaces UVA : Single Address Space System Memory CPU GPU GPU Memory PCI-e 0x0000 0xFFFF 0x0000 0xFFFF System Memory CPU GPU GPU Memory PCI-e 0x0000 0xFFFF
- 6. Unified Virtual Addressing Oneaddress space for all CPU and GPU memory Determine physical memory location from a pointer value Enable libraries to simplify their interfaces (e.g. MPI and cudaMemcpy) Supported on Tesla starting with Fermi 64-bit applications on Linux and Windows TCC Pinned fabric Buffer Host Buffer memcpy GPU Buffer cudaMemcpy Pinned fabric BufferPinned fabric Buffer
- 7. MPI+CUDA //MPI rank 0 MPI_Send(s_buf_d,size,…); //MPIrank n-1 MPI_Recv(r_buf_d,size,…); With UVA and CUDA-aware MPI //MPI rank 0 cudaMemcpy(s_buf_h,s_buf_d,size,…); MPI_Send(s_buf_h,size,…); //MPI rank n-1 MPI_Recv(r_buf_h,size,…); cudaMemcpy(r_buf_d,r_buf_h,size,…); No UVA and regular MPI CUDA-aware MPI makes MPI+CUDA easier.
- 8. CUDA-Aware MPI LibrariesMay Use RDMA to completely remove CPU from the picture. Stage GPU buffers through CPU memory automatically Pipeline messages All the programmer needs to know is they’ve passed a GPU pointer to MPI, the library developer can optimize the rest GPU 0 GPU 1 GPU 0 GPU 1 CPU 0 CPU 1 GPU 0 GPU 1
- 9. GPU-Awareness in CrayMPI Cray began supporting GPU-awareness in 5.6.3 Functions on XK7, but not optimally performing Expected to work very well on XC30 Must be explicitly enabled via run-time environment variable MPICH_RDMA_ENABLED_CUDA Works with both CUDA and OpenACC Version 5.6.4 adds a pipelining feature that should help large messages Enabled with MPICH_G2G_PIPELINE
- 10. OMB Latency Host-to-Host willalways have the lowest latency (fewest hops) Staging through host memory explicitly adds significant latency GPU-aware library is able to fall in the middle. Note: 2 nodes on separate blades.
- 11. OMB Bandwidth Once again,H2H wins out (probably by a difference of latency) Direct RDMA suffers badly with this benchmark. Setting MPICH_G2G_PIPELINE=64 pipelines messages and opens up more concurrency. Hand- Pipelining Came ~Here
- 12. OMB Bandwidth, VaryingPipelines OMB sends messages in a window of 64, so that is naturally optimal. Counter-intuitive that no intermediate values seemed to help. Additionally tried varying chunk sizes with no benefit.
- 13. Optimizing Performance ofa Message MPI vendors know how to optimize performance for an interconnect Different approaches for different message sizes Multiple algorithms Unfortunately, on the XK7, this may not be the optimal approach.
- 14. MPI Lacks theAbility to Express Dependencies One way to negate the cost of G2G communication is to overlap with something computation. Restructuring order of computation may allow such overlapping Some communication patterns have a natural concurrency that isn’t easily exploited. N W E Exchange Cells With N,S,E,W Neighbors. S
- 15. Exploiting Communication Concurrency Thiscannot be expressed in GPU-aware MPI today. Pack North D2H North Transfer North H2D North Unpack North Pack East D2H East Transfer East H2D East Unpack East Pack South D2H South Transfer South H2D South Unpack South Pack West D2H West Transfer West H2D West Unpack West
- 16. Concurrent Messaging Pseudocode doi=N,W MPI_Irecv(i) enddo do i=N,W packBufferAsync(i) copyBufferD2HAsync(i) enddo while(more to send/recv) if Irecv completed copyBufferH2DAsync unpackBufferAsync() endif if D2H completed MPI_Isend() endif done
- 17. Talks Related tothis Optimization HOMME - Matt Norman – CUG2012 S3D - John Levesque – CUG2013
- 18. Summary Optimizing kernels willonly take you so far as you scale, communication cannot be an afterthought. GPU-aware MPI libraries are becoming available Easier to program Can optimize performance of individual message transfers Some communication patterns have a natural concurrency that can be exploited to make communication “free”, but this takes additional effort.