CUDA is a parallel computing platform that allows developers to use GPUs for general purpose processing. It provides a programming model for writing C/C++ applications that leverage the parallel compute engines on Nvidia GPUs. CUDA applications use a data-parallel programming model where the GPU runs many lightweight threads concurrently. The CUDA programming model exposes a hierarchical memory structure including registers, shared memory, and global memory. Developers can write CUDA programs that transfer data from CPU to GPU memory, launch kernels on the GPU, and copy results back to the CPU.

1. COMPUTE
UNIFIED DEVICE
ARCHITECTURE
MANNU MALHOTRA
ENROLL. NO. 08313303110
IT 7TH SEM.

2. CONTENTS
• What is GPU ?
• What is CUDA ?
• Why CUDA ?
• GPU VS CPU.
• DATA FLOW IN CUDA
• Programming model
• CUDA Memory Hierarchy
• A simple program in CUDA
• CUDA Limitations

3. WHAT IS GPU ?
• A GPU is a processor with thousands of cores , ALUs and cache.

4. GPU ARE SO MUCH POWERFUL THAN WHY COMPILER
RUNS OUR PROGRAM ON CPU?
The programming paradigm is not meant for GPU.
CUDA

5. WHAT IS CUDA?
• CUDA is a set of developing tools to create applications that will
perform execution on GPU (Graphics Processing Unit).
• CUDA compiler uses variation of C with future support of C++.
• CUDA was developed by NVidia and can only run on NVidia GPUs of
tesla and Geforce series.
• CUDA provides Heterogeneous serial-parallel computing Between
CPU and GPU

6. WHY CUDA ?
• CUDA provides ability to use high-level languages such as C to
develop application that can take advantage of high level performance
and scalability that GPUs architecture offer.
• GPUs allow creation of very large number of concurrently executed
threads at very low system resource cost.
• CUDA also exposes fast shared memory (16KB) that can be shared
between threads.
• Full support for integer and bitwise operations.
• Compiled code will run directly on GPU.

7. GPU VS. CPU
• GPUs contain much larger number
of dedicated ALUs.
• GPUs also contain extensive
support of Stream Processing
paradigm. It is related to SIMD (
Single Instruction Multiple Data)
processing.
• Each processing unit on GPU
contains local memory that
improves data manipulation and
reduces fetch time.

8. COMPUTATION TIME (CPU VS GPU)

9. FLOW OF DATA
PCIE
1. Data moved from CPU to GPU.
2. Launch Program on GPU.
3. Copy final values to CPU.

10. PROGRAMMING MODEL

11. CUDA MEMORY HIERARCHY
• The CUDA platform has three primary memory types
Local Memory – per thread memory for automatic variables and
register spilling.
Shared Memory – per block memory to allow for intra-block data
sharing and synchronization. Threads can safely share data through
this memory and can perform barrier synchronization through _
_syncthreads()
Global Memory – device level memory that may be shared between
blocks or grids

12. A VECTOR ADD EXAMPLE IN C

13. VECTOR ADD IN CUDA C

14. CUDA LIMITATIONS
• No support of recursive function. Any recursive function must be
converted into loops.
• Bus bandwidth and latency between GPU and CPU is a bottleneck
for many applications.
• Threads should only be run in groups of 32 and up for best
performance.
• Only supported on NVidia GPUs

15. REFERENCES
• Video tutorials by-
• Udacity.com by john owens , Professor of electric and computer at UC Davis.
• Plural site by Dimitri Nesteruk
• Web resources –
• Developer.nvidia.com
• Nividia.com/cuda

16. THANK YOU