This document discusses GPU algorithms and graphics processing units (GPUs). It provides an overview of using GPUs for general purpose tasks by copying data to the GPU memory, running executable kernels on the GPU, and copying results back to the CPU memory. It also discusses GPU memory management and optimizations like using shared memory and avoiding serialization. An example k-means clustering algorithm is mentioned to illustrate a potential GPU algorithm.

1. GPU Algorithms
David Hauck
github.com/davidhauck
@david_hauck_mke
davidhauck40.blogspot.com
dhauck@skylinetechnologies.com

2. Graphics Processing
Unit

5. Why?

7. Graphics Processing
Unit

8. Graphics Processing
Unit
General Purpose

9. T E
M S
R

10. HOST

11. DEVICE

13. PCI Bus
Copy initial
data to
DEVICE

14. PCI Bus
Run DEVICE
Executable

15. PCI Bus
Copy Results
Back To HOST

16. Still Running on CPU

17. Still Running on CPU
GPU is a Resource

19. MEMORY
CONSCIOUSNESS

20. HOST DEVICE
POINTERSPOINTERS

21. int *a;

22. int *a;
int *d_a;

23. arr = malloc(size);

24. arr = malloc(size);
cudaMalloc(&d_arr, size);

25. free(arr);

26. free(arr);
cudaFree(d_arr);

27. memcpy(dest, source, size);

28. memcpy(dest, source, size);
cudaMemcpy(&dest, src, size, …);

29. 1: HOST DEVICE
2: EXECUTE
3: DEVICE HOST

30. 1: HOST DEVICE
3: DEVICE HOST
cudaMemcpy();

31. 1: HOST DEVICE
cudaMemcpy(
&dest,
source,
size, ..hostToDevice);

32. EXECUTION

33. __global__ void myKernel(int *a){
}

34. myKernel<<<1,1>>>(d_arr);

35. Let’s do an example

36. a
b
c
d
+
e
f
g
h
=
i
j
k
l

37. a
b
c
d
+
e
f
g
h
=
i
j
k
l

38. a
b
c
d
+
e
f
g
h
=
i
j
k
l
threadIdx.x
0
1
2
3

39. int index = threadIdx.x;
c[index] =
a[index] +
b[index];

40. Let’s invent an
ALGORITHM

41. K-Means Clustering

55. CODE

56. Shared Memory
• ~48k
• Multiple GB device memory (100x higher latency)
• Access memory in order
• 1 2 3
• 4 5 6
• 7 8 9

57. Considerations
• Transistors are allocated to arithmetic, not memory. Sometimes
better to recompute rather than cache
• Copying to/from host takes a while. Sometimes sequential operations
can stay on gpu
• Avoid serialization (shared memory bank conflicts)
• Asynchronous memory operations