IAP09 CUDA@MIT 6.963 - Lecture 05: Wheel Of Reincarnation (Steven Johnson, MIT)

•

1 like•677 views

npinto

More at http://sites.google.com/site/cudaiap2009 and http://pinto.scripts.mit.edu/Classes/CUDAIAP2009

Education

The Wheel Today

1.5 turns around the wheel?
the display processor on the display processor

More than one turn around the wheel?

stream processors → raster processor → display chip

The conclusion in 2008?
Asymmetric/heterogeneous multiprocessing is hard.
(multiple memory models, different CPU characteristics)

But…serial CPUs aren’t getting faster
But…most programs are serial and need a fast serial CPU
But…1000 Pentium M’s (Intel Core) aren’t coming to desktops

Natural tendency to few beefy CPUs + zillions of simple ones…
GPGPU (Nvidia Tesla/AMD Firestream)?
Cell processors?
Intel Larrabee?
Something else?

More from npinto

[Harvard CS264] 16 - Managing Dynamic Parallelism on GPUs: A Case Study of Hi...

npinto

[Harvard CS264] 15a - The Onset of Parallelism, Changes in Computer Architect...

npinto

[Harvard CS264] 15a - Jacket: Visual Computing (James Malcolm, Accelereyes)

npinto

[Harvard CS264] 14 - Dynamic Compilation for Massively Parallel Processors (G...

npinto

[Harvard CS264] 13 - The R-Stream High-Level Program Transformation Tool / Pr...

npinto

[Harvard CS264] 12 - Irregular Parallelism on the GPU: Algorithms and Data St...

npinto

[Harvard CS264] 11b - Analysis-Driven Performance Optimization with CUDA (Cli...

npinto

[Harvard CS264] 11a - Programming the Memory Hierarchy with Sequoia (Mike Bau...

npinto

[Harvard CS264] 10b - cl.oquence: High-Level Language Abstractions for Low-Le...

npinto

http://cs264.org Abstract: High-level scripting languages are in many ways polar opposites to GPUs. GPUs are highly parallel, subject to hardware subtleties, and designed for maximum throughput, and they offer a tremendous advance in the performance achievable for a significant number of computational problems. On the other hand, scripting languages such as Python favor ease of use over computational speed and do not generally emphasize parallelism. PyOpenCL and PyCUDA are two packages that attempt to join the two together. By showing concrete examples, both at the toy and the whole-application level, this talk aims to demonstrate that by combining these opposites, a programming environment is created that is greater than just the sum of its two parts. Speaker biography: Andreas Klöckner obtained his PhD degree working with Jan Hesthaven at the Department of Applied Mathematics at Brown University. He worked on a variety of topics all aiming to broaden the utility of discontinuous Galerkin (DG) methods. This included their use in the simulation of plasma physics and the demonstration of their particular suitability for computation on throughput-oriented graphics processors (GPUs). He also worked on multi-rate time stepping methods and shock capturing schemes for DG. In the fall of 2010, he joined the Courant Institute of Mathematical Sciences at New York University as a Courant Instructor. There, he is working on problems in computational electromagnetics with Leslie Greengard. His research interests include: - Discontinuous Galerkin and integral equation methods for wave propagation - Programming tools for parallel architectures - High-order unstructured particle-in-cell methods for plasma simulation

[Harvard CS264] 10a - Easy, Effective, Efficient: GPU Programming in Python w...

npinto

Abstract: Machine learning researchers and practitioners develop computer algorithms that "improve performance automatically through experience". At Google, machine learning is applied to solve many problems, such as prioritizing emails in Gmail, recommending tags for YouTube videos, and identifying different aspects from online user reviews. Machine learning on big data, however, is challenging. Some "simple" machine learning algorithms with quadratic time complexity, while running fine with hundreds of records, are almost impractical to use on billions of records. In this talk, I will describe lessons drawn from various Google projects on developing large scale machine learning systems. These systems build on top of Google's computing infrastructure such as GFS and MapReduce, and attack the scalability problem through massively parallel algorithms. I will present the design decisions made in these systems, strategies of scaling and speeding up machine learning systems on web scale data. Speaker biography: Max Lin is a software engineer with Google Research in New York City office. He is the tech lead of the Google Prediction API, a machine learning web service in the cloud. Prior to Google, he published research work on video content analysis, sentiment analysis, machine learning, and cross-lingual information retrieval. He had a PhD in Computer Science from Carnegie Mellon University.

[Harvard CS264] 09 - Machine Learning on Big Data: Lessons Learned from Googl...

npinto

[Harvard CS264] 08a - Cloud Computing, Amazon EC2, MIT StarCluster (Justin Ri...

npinto

[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)

npinto

[Harvard CS264] 07 - GPU Cluster Programming (MPI & ZeroMQ)

npinto

[Harvard CS264] 06 - CUDA Ninja Tricks: GPU Scripting, Meta-programming & Aut...

npinto

[Harvard CS264] 05 - Advanced-level CUDA Programming

npinto

[Harvard CS264] 04 - Intermediate-level CUDA Programming

npinto

[Harvard CS264] 03 - Introduction to GPU Computing, CUDA Basics

npinto

[Harvard CS264] 02 - Parallel Thinking, Architecture, Theory & Patterns

npinto

[Harvard CS264] 01 - Introduction

npinto

More from npinto (20)