This document provides an overview of CUDA architecture and programming. It discusses key CUDA concepts like the host/device model, CUDA C extensions, GPU memory management, and parallel programming using CUDA threads and blocks. CUDA allows developers to speed up applications by offloading work to the GPU. It provides a scalable parallel programming model that maps threads to GPU threads to express data-level parallelism across thousands of lightweight threads for applications like high-bandwidth computing and visual computing.

1. CUDAArchitecture
Prof. Shashikant V. Athawale
Assistant Professor | Computer Engineering
Department | AISSMS College of Engineering,
Kennedy Road, Pune , MH, India - 411001

2. Contents
❖ CUDAArchitecture
❖ Applications of CUDA
❖ Introduction to CUDA C-Write and launch CUDA C
kernels
❖ Manage GPU memory
❖ Manage communication and synchronization
❖ Parallel programming in CUDA- C.

3. Communication And Synchronization in
Thread

4. Communication And Synchronization in
Thread

5. CUDAArchitecture

6. CUDAArchitecture

7. Applications of CUDA

8. CUDA C : The Basics
❖ Based on industry-standard C
❖ A handful of language extensions to allow heterogeneous
programs
❖ Straightforward APIs to manage devices, memory, etc.
❖ Terminology:
➢ Host – The CPU and its memory (host memory)
➢ Device – The GPU and its memory (device memory)
Device

9. CUDA Kernels

10. GPU Memory Management

11. Data Transfer Directions Keywords
❖ cudaMemcpyHostToHost
❖ cudaMemcpyHostToDevice
❖ cudaMemcpyDeviceToHost
❖ cudaMemcpyDeviceToDevice

12. Parallel Programming in CUDA C
❖ CUDA brings data-parallel computing to the masses.
❖ CUDA is a scalable parallel programming model.
❖ Program runs on any number of processors without
recompiling.

13. Architecture Of Parallel CUDA Programming

14. CUDA Uses Extensive Multithreading
❖ CUDA threads express fine-grained data parallelism.
➢ Map threads to GPU threads.
➢ Virtualize the processors.
❖ CUDA thread blocks express coarse-grained parallelism.
➢ Blocks hold arrays of GPU threads, define shared
memory boundaries.
➢ Allow scaling between smaller and larger GPUs.

15. CUDA Uses Extensive Multithreading
❖ GPUs execute thousands of lightweight threads.
➢ In graphics, each thread computes one pixel.
➢ One CUDA thread computes one result (or several
results).
➢ Hardware multithreading & zero-overhead
scheduling.

16. Applications
❖ High bandwidth
❖ Visual computing
❖ High arithmetic intensity