GPUs are highly parallel and programmable processors that were originally designed for graphics processing but are now commonly used for general purpose computing. GPU power is increasing much faster than CPU power, with estimates of a 570x increase in GPU power versus a 3x increase in CPU power over 6 years. GPUs excel at applications with large datasets, high parallelism, and minimal data dependencies, such as molecular dynamics, physics simulations, and database operations. Programming models for GPUs follow a single-program multiple-data approach to maximize parallel processing across many cores.

1. GPU COMPUTING
Presented By
Rajiv Kumar V
No -34
S7C

2. Graphics Processing Units(GPU):
Powerful
Programmable and
Highly Parallel
Jen-sun Huang ,
” GPU power is set to increase 570x whereas CPU
power would increase a mere 3x over the same time
frame of six years”

3. INTRODUCTION:
• GPU has powered the display of Computers
• Designed for real-time high resolution 3D graphics tasks
• Commercial GPU-based systems are becoming common
• NVIDIA and AMD expanding processor sophistication and
software development tools
• High accuracy by higher floating point precision
• GPUs currently on a development cycle much closer to CPUs
• GPU not constrained by sockets
• Very small backwards compatibility needed in firmware while
rest is delivered through driver implementation

4. GPU based S/W’s requirements
• Computational requirements are large
• Parallelism is substantial
• Throughput is more important than latency
App requirement to target GPGPU
programming:
• Large data sets
• High parallelism
• Minimal dependencies between data elements
• High arithmetic intensity
• Lots of work to do without CPU intervention

5. Task Vs. Data parallelism
Task parallelism:
• Independent processes with little
communication
Data parallelism:
• Lots of data on which the same computation is
being executed
• No dependencies between data elements in
each step in the computation
• Can saturate many ALUs

6. GPU Vs CPU
• CPU designed to process a task as fast as
possible while GPU capable of processing a
maximum of tasks on a large scale of data
• CPU divides work in time while GPU divides work
in space

7. Graphics Pipeline:
• Input to the GPU is a list of geometric primitives
• Vertex Operations: primitives transformed into screen
space and shaded
• Primitive Assembly: Vertices assembled into triangles
• Computing their interaction with the lights in the scene
• Rasterization: determines which screen-space pixels are
covered by each triangle
• Fragment Operations: Using color information each
fragment is shaded to determine its final color
• Each pixel’s color value may be computed from several
fragments
• Composition: Fragments are assembled into a final image
with one color per pixel

8. Graphics Pipeline:

9. Evolution of GPU Architecture:
• Fixed function pipeline lacked generality for complex
effects
• Replacement of fixed function per vertex and per
fragment operations by vertex and fragment programs
• Increased complexity of vertex and fragment program
as Shader Model evolved
• Support for Unified Shader Models
Shader Models:
• A Shader provides a user defined programmable
alternative to hard-coded approach in GLSL
• A Vertex Shader describe the traits(position, colors ,
depth value etc) of a vertex
• A Geometry shader add volumetric detail & O/P is then
sent to the rasterizer
• A Pixel/fragment shader describe the traits (color, z-
depth and alpha value) of a pixel

11. GPU Programming Model
• Follows a SPMD programming model
• Each element is independent from other
elements in base programming Model
• Many parallel elements processed by single
program
• Each element can operate on integer or
float data with reasonably complete
instruction set
• Reads data from shared memory by scatter
and gather operations
• Code is in SIMD manner
• Allows different execution path for each
element
• If elements branch in different directions
both branches are computed
• Computation as blocks in order of 16
elements
• Finally programmers branches are
permitted but not free

12. GPU Architecture:NVIDIA
Nvidia 8800GTX architecture
(top)
A pair of SMs(right)

13. Memory Architecture
• Capable of reading and writing anywhere in local
memory(GPU) or elsewhere.
• These non cached memories having large read/write
latencies which can be masked by the extremely long
pipeline, if they don’t wait for a reading instruction

14. GPGPU Programming
Stream processing is a new paradigm to maximize the
efficiency of parallel computing. It can be decomposed in two
parts:
• Stream: It’s a collection of objects which can be operated
in parallel and which require the same computation.
• Kernel: It’s a function applied on the entire stream, looks
like a “for each” loop

15. Terminology:
Streams
-Collection of records requiring similar computation
eg. Vertex positions, Voxels etc.
-Provide data parallelism
Kernels
–Functions applied to each element in stream
transforms
–No dependencies between stream elements
encourage high Arithmetic Intensity
Gather
–Indirect read from memory ( x = a[i] )
–Naturally maps to a texture fetch
–Used to access data structures and data streams
Scatter
–Indirect write to memory ( a[i] = x )
–Needed for building many data structures
–Usually done on the CPU

16. What can you do on GPUs other than
graphics?
• Large matrix/vector operations (BLAS)
• Protein Folding (Molecular Dynamics)
• FFT (SETI, signal processing)
• Ray Tracing
• Physics Simulation [cloth, fluid, collision]
• Sequence Matching (Hidden Markov Models)
• Speech Recognition (Hidden Markov
Models, Neural nets)
• Databases
• Sort/Search
• Medical Imaging (image
segmentation, processing)
And many, many more…

17. Future of GPU Computing:
• Higher Bandwidth PCI-E bus path between
CPU and GPU
• AMD’s fusion and Intel’s IvyBridge places
both CPU and GPU elements on a single chip
• Addition of AVX instructions in CPU
architectures
• Programmable Pipelines over the current few
programmable shading stages in the fixed
graphics pipeline
• Flexibility of variety of rendering along with
general purpose processing

18. Looking Ahead:

19. Problems in GPGPU Computing
• A killer App...???...??
• Programming models and Tools…Proprietary
nature…??
• GPU in tomorrow’s Computer…Will it get
dissolved…or absorbed???
• Relationship to other parallel H/W and S/W
• Managing Rapid Change…
• Performance Evaluation and Cliffs
• Broader Toolbox for computation and Data
Structures…”Vertical” model for app development
• Faults and Lack of Precision…

20. Drawbacks:
• Power consumption
• Increasing die size
• Multi die solutions requiring inter-die connections
increase the packaging and wafer cost
• Increasing amount of die space to control logic
, registers and cache as GPU becomes flexible and
programmable
• Comparing CPU to GPUs is more like comparing
apples to oranges
• Still lots of fixed functions hardware
• Integration of multimedia fixed functions within
the CPUs

21. References:
• GPU Computing Gems Emerald Edition By Wen.Mei W.
Hwu
• Cuda By Example: An Introduction to General
Purpose GPU Computing By J.Sanders,E.Kandrot (July
2010)
• http://www.oxford-man.ox.ac.uk/gpuss/simd.html
• http://idlastro.gsfc.nasa.gov/idl_html_help/About_Sh
ader_Programs.html
• GPU Computing Proceedings of IEEE,May 2008
• Evolution Of GPU By Chris Sietz

22. Thank You All…
Any Questions…
???