This document discusses the shift towards parallel computing due to physical limitations in processor speed improvements. It introduces MapReduce as a programming model for easily writing parallel programs to process large datasets across many computers. MapReduce works by splitting data, processing it in parallel via mapping functions, then collecting the results via reducing functions. Examples show how it can be used to count word frequencies or crawl the web in parallel.

1. Invent Show Tech Talk SeriesParallel Future

2. Parallelism is Here…….12-Aug-10Invent Show2In the words of Sun Microsystems researcher “Guy Steele”:“ The bag of programming tricks that has served us so well for the last 50 years is the wrong way to think going forward and must be thrown out.”In the words of famous Berkeley Professor “Dave Patterson”:“ We desperately need new approach to hardware and software based on parallelism since industry has bet its future that parallelism works”

3. The Paradigm Shift – What Caused It?12-Aug-10Invent Show3Moore’s Law:“The density of transistors on a chip doubles every 18 months, for the same cost.”Now failedWe have reached a limit in reducing the transistor size – Power WallMemory bandwidth is now an issue – Memory WallSet of problems we can solve with a single computer is not going to get any larger – ILP WallSolution:Parallel computing – multicoresDistributed computing – data centers (Google, Facebook, Yahoo)

4. So What is the Difference?Good sequential codeGood Parallel Code12-Aug-10Invent Show4Minimizes total number of operations.Minimizes space usage.Stresses linear problem decomposition.Performs redundant operations.Requires extra space.Requires multiway problem decomposition.

5. Basics12-Aug-10Invent Show5Not all code can be parallelizedFibonacci function: Fk+2= Fk+ Fk+1But most of the computations can be parallelizedLarge amount of consistent data to be processed with no dependencies

6. Basic Model – Master/Worker Model (1/2)12-Aug-10Invent Show6Consider a huge array that can be broken into sub-arrays

7. Basic Model – Master/Worker Model (2/2)12-Aug-10Invent Show7MASTERInitializes the array and splits it up according to the number of WORKERSSends each WORKER its subarrayReceives the results from each WORKERWORKERReceives the subarray from the MASTERPerforms processing on the subarrayReturns results to MASTER

8. MapReduce12-Aug-10Invent Show8Simple data-parallel programming model designed for scalability and fault-tolerancePioneered by GoogleProcesses 20 petabytes of data per dayPopularized by open-source Hadoop projectUsed at Yahoo!, Facebook, Amazon, …

9. What is MapReduce used for? (1/2)12-Aug-10Invent Show9At Google:Index construction for Google SearchArticle clustering for Google NewsStatistical machine translationAt Yahoo!:“Web map” powering Yahoo! SearchSpam detection for Yahoo! MailAt Facebook:Data miningAd optimizationSpam detection

10. What is MapReduce used for? (2/2)12-Aug-10Invent Show10In research:Astronomical image analysis (Washington)Bioinformatics (Maryland)Analyzing Wikipedia conflicts (PARC)Natural language processing (CMU) Particle physics (Nebraska)Ocean climate simulation (Washington)VisionerBOT – our custom Web crawler

11. MapReduce Programming Model12-Aug-10Invent Show11Data type: key-value recordsMap function:(Kin, Vin)  list(Kinter, Vinter)Reduce function:(Kinter, list(Vinter))  list(Kout, Vout)

12. Example: Word Count12-Aug-10Invent Show12def mapper(line):foreach word in line.split(): output(word, 1)def reducer(key, values): output(key, sum(values))

13. Word Count Execution12-Aug-10Invent Show13ReduceOutputInputMapShuffle & Sortthe, 1brown, 1fox, 1the quickbrown foxbrown, 2fox, 2how, 1now, 1the, 3MapReducethe, 1fox, 1the, 1the fox atethe mouseMapquick, 1how, 1now, 1brown, 1ate, 1cow, 1mouse, 1quick, 1ate, 1mouse, 1Reducehow nowbrown cowMapcow, 1

14. Example: VisionerBot Web Crawler 12-Aug-10Database and Multimedia Lab.14

15. MapReduce Execution Details12-Aug-10Invent Show15Single master controls job execution on multiple slavesThere could be hierarchy of masters under the control of absolute master Mappers are preferably placed near to each other in order to minimize network delayThere should be checkpoints to make sure recovery process if some operation gets crashed

16. 12-Aug-10Invent Show16QUESTIONS AND FEEDBACK