The document discusses GPU architecture and computing. It describes how GPUs have evolved from specialized graphics processors into general parallel computing devices. The document outlines GPU architecture including the graphics pipeline, use of shader programs, and SPMD programming model. It also discusses virtualizing GPUs through front-end and back-end approaches to enable sharing of physical GPUs across VMs.

2. INTRODUCTION
• THE GRAPHICS PROCESSING UNIT (GPU) HAS BECOME AN INTEGRAL PART OF
TODAY’S MAINSTREAM COMPUTING SYSTEMS. OVER THE PAST SIX YEARS, THERE
HAS BEEN A MARKED INCREASE IN THE PERFORMANCE AND CAPABILITIES OF
GPUS.
• GPU IS A GRAPHICAL PROCESSING UNIT WHICH ENABLES YOU RUN HIGH
DEFINITIONS GRAPHICS ON YOUR PC, WHICH ARE THE EMEND OF MODERN
COMPUTING. LIKE THE CPU (CENTRAL PROCESSING UNIT), IT IS A SINGLE-CHIP
PROCESSOR. THE GPU HAS HUNDREDS OF CORES AS COMPARED TO THE 4 OR 8 IN
THE LATEST CPUS. THE PRIMARY JOB OF THE GPU IS TO COMPUTE 3D FUNCTIONS.

3. PICTURES OF THE GPU
Nvidia Geforce GTX 1070 FTW 8 GB

4. AMD Radeon R9 295X2

5. GPU ARCHITECTURE
• CONTROL HARDWARE DOMINATES PROCESSORS
• COMPLEX, DIFFICULT TO BUILD AND VERIFY
• TAKES SUBSTANTIAL FRACTION OF DIE SCALES POORLY
• PAY FOR MAX THROUGHPUT, SUSTAIN AVERAGE THROUGHPUT
• QUADRATIC DEPENDENCY CHECKING
• CONTROL HARDWARE DOESN’T DO ANY MATH!
• OVER THE PAST FEW YEARS, THE GPU HAS EVOLVED FROM A FIXED-FUNCTION
SPECIAL-PURPOSE PROCESSOR INTO A FULL-FLEDGED PARALLEL PROGRAMMABLE
PROCESSOR WITH ADDITIONAL FIXED-FUNCTION SPECIAL-PURPOSE
FUNCTIONALITY.

6. GPU ARCHITECTURE
• THE GRAPHICS PIPELINE
- THE INPUT TO THE GPU IS A LIST OF GEOMETRIC PRIMITIVES,
TYPICALLY TRIANGLES, IN A 3-D WORLD COORDINATE SYSTEM. THROUGH MANY
STEPS, VERTEX OPERATIONS: THE INPUT PRIMITIVES ARE FORMED FROM
INDIVIDUAL VERTICES. EACH VERTEX MUST BE TRANSFORMED INTO SCREEN
SPACE AND SHADED, TYPICALLY THROUGH COMPUTING THEIR INTERACTION
WITH THE LIGHTS IN THE SCENE. BECAUSE TYPICAL SCENES HAVE TENS TO
HUNDREDS OF THOUSANDS OF VERTICES, AND EACH VERTEX CAN BE COMPUTED
INDEPENDENTLY, THIS STAGE IS WELL SUITED FOR PARALLEL HARDWARE.

7. GPU ARCHITECTURE

8. GPU ARCHITECTURE
• EVOLUTION OF GPU ARCHITECTURE
- THE FIXED-FUNCTION PIPELINE LACKED THE GENERALITY TO
EFFICIENTLY EXPRESS MORE COMPLICATED SHADING AND LIGHTING
OPERATIONS THAT ARE ESSENTIAL FOR COMPLEX EFFECTS. THE KEY STEP WAS
REPLACING THE FIXED-FUNCTION PER-VERTEX AND PER-FRAGMENT OPERATIONS
WITH USER-SPECIFIED PROGRAMS RUN ON EACH VERTEX AND FRAGMENT. OVER
THE PAST SIX YEARS, THESE VERTEX PROGRAMS AND FRAGMENT PROGRAMS
HAVE BECOME INCREASINGLY MORE CAPABLE, WITH LARGER LIMITS ON THEIR
SIZE AND RESOURCE CONSUMPTION, WITH MORE FULLY FEATURED
INSTRUCTION SETS, AND WITH MORE FLEXIBLE CONTROL-FLOW OPERATIONS.

9. GPU ARCHITECTURE
• ARCHITECTURE OF MODERN GPU
- WE NOTED THAT THE GPU IS BUILT FOR DIFFERENT APPLICATION
DEMANDS THAN THE CPU: LARGE PARALLEL COMPUTATION REQUIREMENTS WITH
AN EMPHASIS ON THROUGHPUT RATHER THAN LATENCY. CONSEQUENTLY, THE
ARCHITECTURE OF THE GPU HAS PROGRESSED IN A DIFFERENT DIRECTION THAN
THAT OF THE CPU.
- THE CPU DIVIDES THE PIPELINE IN TIME, APPLYING ALL RESOURCES IN
THE PROCESSOR TO EACH STAGE IN TURN. GPUS HAVE HISTORICALLY TAKEN A
DIFFERENT APPROACH. THE GPU DIVIDES THE RESOURCES OF THE PROCESSOR
AMONG THE DIFFERENT STAGES, SUCH THAT THE PIPELINE IS DIVIDED IN SPACE, NOT
TIME. THE PART OF THE PROCESSOR WORKING ON ONE STAGE FEEDS ITS OUTPUT
DIRECTLY INTO A DIFFERENT PART THAT WORKS ON THE NEXT STAGE.

10. GPU ARCHITECTURE

11. LATEST TECHNOLOGIES USED
• NVIDIA
– TESLA HPC SPECIFIC GPUS HAVE EVOLVED FROM GEFORCE SERIES
• AMD
– FIRE STREAM HPC SPECIFIC GPUS HAVE EVOLVED FROM (ATI)
RADEON SERIES
• INTEL
– KNIGHTS CORNER MANY-CORE X86 CHIP IS LIKE HYBRID BETWEEN
A GPU AND MANY-CORE CPU

12. CHARACTERISTICS OF
GRAPHICS IN GPU
• LARGE COMPUTATIONAL REQUIREMENTS
• MASSIVE PARALLELISM
• GRAPHICS PIPELINE DESIGNED FOR INDEPENDENT OPERATIONS
• LONG LATENCIES TOLERABLE
• DEEP, FEED-FORWARD PIPELINES
• HACKS ARE OK—CAN TOLERATE LACK OF ACCURACY
• GPUS ARE GOOD AT PARALLEL, ARITHMETICALLY INTENSE, STREAMING-
MEMORY PROBLEMS

13. ASSUMPTIONS AND
LIMITATIONS IN GPU
• IN ADDITION TO QUERY PROCESSING, LARGE WEB SEARCH ENGINES NEED TO
PERFORM MANY OTHER OPERATIONS INCLUDING WEB CRAWL-ING, INDEX
BUILDING, AND DATA MINING STEPS FOR TASKS SUCH AS LINK ANALYSIS AND
SPAM AND DUPLICATE DETECTION. WE FOCUS HERE ON QUERY PROCESSING AND
IN PARTICULAR ON ONE PHASE OF THIS STEP AS EXPLAINED FURTHER BELOW. WE
BELIEVE THAT THIS PART IS SUITABLE FOR IMPLEMENTATION ON GPUS AS IT IS
FAIRLY SIMPLE IN STRUCTURE BUT NONETHELESS CONSUMES A
DISPROPORTIONATE AMOUNT OF THE OVERALL SYSTEM RESOURCES. IN
CONTRAST, WE DO NOT THINK THAT IMPLEMENTATION OF A COMPLETE SEARCH
ENGINE ON A GPU IS CURRENTLY REALISTIC.

14. GPU COMPUTING
• THE GPU PROGRAMMING MODEL
- THE PROGRAMMABLE UNITS OF THE GPU FOLLOW A SINGLE
PROGRAM MULTIPLE-DATA (SPMD) PROGRAMMING MODEL. FOR EFFICIENCY,
THE GPU PROCESSES MANY ELEMENTS (VERTICES OR FRAGMENTS) IN PARALLEL
USING THE SAME PROGRAM. EACH ELEMENT IS INDEPENDENT FROM THE OTHER
ELEMENTS, AND IN THE BASE PROGRAMMING MODEL, ELEMENTS CANNOT
COMMUNICATE WITH EACH OTHER. ALL GPU PROGRAMS MUST BE STRUCTURED
IN THIS WAY: MANY PARALLEL ELEMENTS, EACH PROCESSED IN PARALLEL BY A
SINGLE PROGRAM.

15. GPU COMPUTING
• GENERAL-PURPOSE COMPUTING ON THE GPU
- STEPS TO SHOW THE SIMPLER AND DIRECT WAY THAT TODAY’S GPU COMPUTING
APPLICATIONS ARE WRITTEN.
1. PROGRAMMING A GPU FOR GRAPHICS: WE BEGIN WITH THE SAME GPU PIPELINE THAT WE
DESCRIBED IN SECTION II, CONCENTRATING ON THE PROGRAMMABLE ASPECTS OF THIS PIPELINE.
2. THE PROGRAMMER SPECIFIES GEOMETRY THAT COVERS A REGION ON THE SCREEN. THE
RASTERIZER GENERATES A FRAGMENT AT EACH PIXEL LOCATION COVERED BY THAT GEOMETRY.
3. EACH FRAGMENT IS SHADED BY THE FRAGMENT PROGRAM.
4. THE FRAGMENT PROGRAM COMPUTES THE VALUE OF THE FRAGMENT BY A COMBINATION OF MATH
OPERATIONS AND GLOBAL MEMORY READS FROM A GLOBAL BTEXTURE [MEMORY].
5. THE RESULTING IMAGE CAN THEN BE USED AS TEXTURE ON FUTURE PASSES THROUGH THE
GRAPHICS PIPELINE.

16. GPU VIRTUALIZATION TAXONOMY
• WE OBSERVE THAT DIFFERENT USE CASES WEIGHT THE CRITERIA
DIFFERENTLY—FOR EXAMPLE A VDI DEPLOYMENT VALUES HIGH VM-TO-GPU
CONSOLIDATION RATIOS (E.G., MULTIPLEXING) WHILE A CONSUMER RUNNING
A VM TO ACCESS A GAME OR CAD APPLICATION UNAVAILABLE ON HIS HOST
VALUES PERFORMANCE AND LIKELY FIDELITY. A TECH SUPPORT PERSON
MAINTAINING A LIBRARY OF DIFFERENT CONFIGURATIONS AND AN IT
ADMINISTRATOR RUNNING SERVER VMS ARE BOTH LIKELY TO VALUE
PORTABILITY AND SECURE ISOLATION (INTERPOSITION).

17. GPU VIRTUALIZATION TAXONOMY
• FRONT-END VIRTUALIZATION
- FRONT-END VIRTUALIZATION INTRODUCES A VIRTUALIZATION BOUNDARY AT A
RELATIVELY HIGH LEVEL IN THE STACK, AND RUNS THE GRAPHICS DRIVER IN THE
HOST/HYPERVISOR. THIS APPROACH DOES NOT RELY ON ANY GPU VENDOR- OR MODEL-
SPECIFIC DE- TAILS.
• BACK-END VIRTUALIZATION
-THE MOST OBVIOUS BACK-END VIRTUALIZATION TECHNIQUE IS FIXED PASS-
THROUGH: THE PERMANENT ASSOCIATION OF A VIRTUAL MACHINE WITH FULL EXCLUSIVE
ACCESS TO A PHYSICAL GPU. RECENT CHIPSET FEATURES, SUCH AS INTEL’S VT-D, MAKE FIXED
PASS-THROUGH PRACTICAL WITHOUT REQUIRING ANY SPECIAL KNOWLEDGE OF A GPU’S
PROGRAMMING INTERFACES. HOWEVER, FIXED PASS-THROUGH IS NOT A GENERAL SOLUTION.
IT COMPLETELY FORGOES ANY MULTIPLEXING AND PACKING MACHINES WITH ONE GPU PER
VIRTUAL MACHINE (PLUS ONE FOR THE HOST) IS NOT FEASIBLE.