IAP09 CUDA@MIT 6.963 - Guest Lecture: Out-of-Core Programming with NVIDIA's CUDA (Gene Cooperman, NEU)

Out-of-Core Programming with NVIDIA’s CUDA Gene Cooperman High Performance Computing Lab College of Computer and Information Science Northeastern University Boston, Massachusetts 02115 USA gene@ccs.neu.edu

Pencil and Paper Calculation • GeForce 8800: – 16 CPU chips/Streaming Multiprocessors (SMs), 8 Cores per chip : 128 cores – Aggregate bandwidth to off-chip global memory: 86.4 GB/s (optimal) – Average bandwidth to global memory per core: 0.67 GB/s • Motherboard – 4 CPU cores – About 10 GB/s bandwidth to main RAM – Average bandwidth to RAM per core: 2.5 GB/s

Keeping Pipe to Memory Flowing • Thread block: threads on a single chip • Thread block organized into warps • Warp of 32 threads required (minimize overhead of switching thread blocks) • Highest bandwidth when all SMs executing same code

Memory-Bound Computations • So, how much data can we keep in the SMs before it overﬂows? • 16 KB/SM −→ 256 KB total cache • Any computation with an active working set of more than 256 KB risks being memory bound.

Memory Bandwidth in Numbers (Thanks to Kapil Arya and Viral Gupta; Illustrative for trends, only) X-Axis: number of thread blocks Y-Axis: bandwidth (MB/s) Different curves: number of threads per thread block.

Is Life Any Better Back on the Motherboard? • Up to 10 GB/s bandwidth to motherboard (perhaps ﬁve times slower than NVIDIA in practice) • Four cores competing for bandwidth • Cache of at least 1 MB, and possibly much more (e.g L3 cache) • Conclusion: Less pressure on memory, but similar order of magnitude

Is Life Any Better between CPU and Disk? • Between 0.05 GB and 0.1 GB bandwidth to disk • Four cores competing for bandwidth • Cache consists of 4 GB or more of RAM • Conclusion: huge pressure on memory (but RAM as cache is large)

Our Solution • Disk is the New RAM • Bandwidth of Disk: ˜ 100 MB/s • Bandwidth of 50 Disks: 50 × 100 MB/s = 5 GB/s • Bandwidth of RAM: approximately 5 GB/s • Conclusion: 1. CLAIM: A computer cluster of 50 quad-core nodes, each with 500 GB of mostly idle disk space, is a good approximation to a shared memory computer with 200 CPU cores and a single subsystem with 25 TB of shared memory. (The arguments also work for a SAN with multiple access nodes, but we consider local disks for simplicity.) 2. The disks of a cluster can serve as if they were RAM. 3. The traditional RAM can then serve as if it were cache.

What About Disk Latency? • Unfortunately, putting 50 disks on it, doesn’t speed up the latency. • So, re-organize the data structures and low-level algorithms. • Our group has ﬁve years of case histories applying this computational algebra — but each case requires months of development and debugging. • We’re now developing both higher level abstractions for run-time libraries, and a language extension that will make future development much faster.

Applications Benefiting from Disk-Based Parallel Computation Discipline Example Application 1. Verification Symbolic Computation using BDDs 2. Verification Explicit State Verification 3. Comp. Group Theory Search and Enumeration in Mathematical Structures 4. Coding Theory Search for New Codes 5. Security Exhaustive Search for Passwords 6. Semantic Web RDF query language; OWL Web Ontology Language 7, Artificial Intelligence Planning 8. Proteomics Protein folding via a kinetic network model 9. Operations Research Branch and Bound 10. Operations Research Integer Programming (applic. of Branch-and-Bound) 11. Economics Dynamic Programming 12. Numerical Analysis ATLAS, PHiPAC, FFTW, and other adaptive software 13. Engineering Sensor Data 14. A.I. Search Rubik’s Cube

Central Claim Suppose one had a single computer with 10 terabytes of RAM and 200 CPU cores. Does that satisfy your need for computers with more RAM? CLAIM: A computer cluster of 32 quad-core nodes, each with a 500 GB local disk, is a good approximation of the above computer. (The arguments also work for a SAN with multiple access nodes, but we discuss local disks for simplicity.)

When is a cluster like a 10 TB shared memory computer? • Assume 200 GB/node of free disk space • Assume 50 nodes, • The bandwidth of 50 disks is 50 × 100MB/s = 5GB/s. • The bandwidth of a single RAM subsystem is about 5GB/s. CLAIM: You probably have the 10 TB of temporary disk space lying idle on your own recent-model computer cluster. You just didn’t know it. (Or were you just not telling other people about the space, so you could use if for yourself?) The economics of disks are such that one saves very little by buying less than 500 GB disk per node. It’s common to buy the 500 GB disk, and reserve the extra space for expansion.

When is a cluster NOT like a 10 TB shared memory computer? 1. We require a parallel program. (We must access the local disks of many cluster nodes in parallel.) 2. The latency problem of disk. 3. Can the network keep up with the disk?

When is a cluster NOT like a 10 TB shared memory computer? . . . and why doesn’t it matter for our purposes? • ANSWER 1: We’ve used this architecture, and it works for us. • We’ve developed solutions for a series of algorithmically simple computational kernels from computational algebra — especially mathematical group theory. All of the following computations completed in less than one cluster-week on a cluster of 60 nodes or less. – Construction of Thompson Sporadic Simple Group (2003) 2 gigabytes (temporary space), 1.4 × 108 states, 4 bytes per state – Construction of Baby Monster Sporadic Simple Group (2006) 6 terabytes (temporary space), 1.4 × 1010 states, 12 bytes per state – Condensation of Fi23 Sporadic Simple Group (2007) 400 GB (temporary space) 1.2 × 1010 states, 30 bytes per state (larger condensation for J4 now in progress) – Rubik’s Cube: 26 Moves Sufﬁce to Solve Rubik’s Cube (2007) 7 terabytes (temporary space), 1012 states, 6 bytes per state – In progress: coset enumeration (pointer-chasing: similar to algorithm for converting NFA to DFA (ﬁnite automata)).

When is a cluster NOT like a 10 TB shared memory computer? 1. We require a parallel program. 2. The latency problem of disk. 3. Can the network keep up with the disk?

When is a cluster NOT like a 10 TB shared memory computer? . . . and why doesn’t it matter for our purposes? 1. We require a parallel program. (We must access the local disks of many nodes in parallel.) • Our bet (still to be proved): Any sequential algorithm that already creates gigabytes of RAM-based data should have a way to create that data in parallel. 2. The latency problem of disk. Solutions exist: (a) For duplicates on frontier in state space search: Delayed Duplicate Detection implies waiting until many nodes of the next frontier (and duplicates from previous iterations) have been discovered. Then remove duplicates. (b) For hash tables, wait until there are millions of hash queries. Then sort on the hash index, and scan the disk to resolve queries. (c) For pointer-chasing, wait until millions of pointers are available for chasing. Then sort and scan the disk to dereference pointers. (d) For tracing strings, with each string being a lookup, wait until millions of strings are available. Then .... 3. Can the network keep up with the disk?

When is a cluster NOT like a 10 TB shared memory computer? . . . and why doesn’t it matter for our purposes? 1. We require a parallel program. (We must access the local disks of many nodes in parallel.) 2. The latency problem of disk. 3. Can the network keep up with the disk? (In our experience to date, the network does keep up. Here are some reasons why it seems to just work.) • The point-to-point bandwidth of Gigabit Ethernet is about 100 MB/s. The bandwidth of disk is about 100 MB/s. As long as the aggregate bandwidth of network can keep up, everything is ﬁne. • Researchers already face the issue of aggregate network bandwidth in RAM-based programs. The disk is slower than RAM. So, probably traditional parallel programs can cope.

Applications from Computational Group Theory (2003–2007) Space State Total Group Size Size Storage 1.17 × 1010 100 bytes Fischer Fi23 1 TB “Baby Monster” 1.35 × 1010 548 bytes 7 TB 1.31 × 1011 64 bytes Janko J4 8 TB (joint with Eric Robinson)

History of Rubik’s Cube • Invented in late 1970s in Hungary. • In 1982, in Cubik Math, Singmaster and Frey conjectured: No one knows how many moves would be needed for “God’s Algorithm” assuming he always used the fewest moves required to restore the cube. It has been proven that some patterns must exist that require at least seventeen moves to restore but no one knows what those patterns may be. Experienced group theorists have conjectured that the smallest number of moves which would be sufﬁcient to restore any scrambled pattern — that is, the number of moves required for “God’s Algorithm” — is probably in the low twenties. • Current Best Guess: 20 moves sufﬁce – States needing 20 moves are known

History of Rubik’s Cube (cont.) • Invented in late 1970s in Hungary. • 1982: “God’s Number” (number of moves needed) was known by authors of conjecture to be between 17 and 52. • 1990: C., Finkelstein, and Sarawagi showed 11 moves suffice for Rubik’s 2 × 2 × 2 cube (corner cubies only) • 1995: Reid showed 29 moves suffice (lower bound of 20 already known) • 2006: Radu showed 27 moves suffice • 2007 Kunkle and C. showed 26 moves suffice • 2008 Rockiki showed 22 moves suffice (using idle resources at Sony Pictures)

Large-Memory Apps: Experience in N.U. Course (mixed undergrads and grads) 1. Chaitin’s Algorithm 2. Fast Permutation Multiplication 3. Kernighan-Lin Partitioning Algorithm 4. Large matrix-matrix Multiplication 5. Voronoi Diagrams 6. Cellular Automata 7. GAA* Search 8. Static Performance Evaluation for Memory Bound Computing Others: [BFS using External Sort] BFS using External Sort [BFS using Segments & Hash Array] BFS using Segments & Hash Array [Fast Permutation Multiplication] Fast Permutation Multiplication [Kernighan-Lin Partitioning Algorithm] Kernighan-Lin Partitioning Algorithm [Large matrix-matrix Multiplication] Large matrix-matrix Multiplication

Example: Rubik’s Cube: Sorting Delayed Duplicate Detection 1. Breadth-first search: storing new frontier (open list) on disk 2. Use Bucket Sorting to sort and eliminate duplicate states from the new frontier (The bucket size is chosen to fit in RAM (the new cache). 3. Storing the new frontier requires 6 terabytes of disk space (and we would use more if we had it). Saving a large new frontier on disk prior to sorting delays duplicate detection, but makes the routine more efficient due to economies of scale.

Rubik’s Cube: Two-Bit trick 1. The ﬁnal representation of the state space (1.4 × 1012 states) could use only 2 bits per state. (We use 4 bits per state for convenience.) 2. We used mathematical group theory to derive a highly dense, perfect hash function (no collisions) for the states of |cube|/|S|. 3. Our hash function represents symmetrized cosets (the union of all symmetric states of |cube|/|S| under the symmetries of the cube). 4. Each hash slot need only store the level in the search tree modulo 3. This allows the algorithm to distinguish states from the current frontier, the next frontier, and the previous frontier (current level; current level plus one; and current level minus one). This is all that is needed.

Space-Time Tradeoffs using Additional Disk • Use even more disk space in order to speed up the algorithm. “A Comparative Analysis of Parallel Disk-Based Methods for Enumerating Implicit Graphs”, Eric Robinson, Daniel Kunkle and Gene Cooperman, Proc. of 2007 International Workshop on Parallel Symbolic and Algebraic Computation (PASCO ’07), ACM Press, 2007, pp. 78–87

LONGER-TERM GOAL: Mini-Language Extension Well-understood building blocks already exist: external sorting, B-trees, Bloom filters, Delayed Duplicate Detection, Distributed Hash Trees (DHT), and some still more exotic algorithms. GOAL: Provide language extensions for common data structures and algorithms (including breadth-first search) that invoke a run-time library. Design the language to bias the programmer toward efficient use of disk. ROOMY LANGUAGE: New Parallel Disk-Based Language, Roomy, in development by Daniel Kunkle. Implementation: Run-time C library with #define and typedef for nicer syntax. Language appears to be sequential; back-end based on cluster with local disks; or cluster with SAN; or single computer using RAM (for simpler development and debugging) Expected availability: mid-2009

IAP09 CUDA@MIT 6.963 - Guest Lecture: Out-of-Core Programming with NVIDIA's CUDA (Gene Cooperman, NEU)

by npinto

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 10

Statistics

IAP09 CUDA@MIT 6.963 - Guest Lecture: Out-of-Core Programming with NVIDIA’s CUDA (Gene Cooperman, NEU) Presentation Transcript

http://www.slideshare.net	9
http://webcache.googleusercontent.com	1