Slideshow Transcript

1. Slide 1: A Presentation on Parallel Computing -Ameya Waghmare(Rno 41,BE CSE) Guided by-Dr.R.P.Adgaonkar(HOD),CSE Dept.
2. Slide 3: • Parallel computing is a form of computation in which many instructions are carried out simultaneously operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently (in parallel). Why is it required?
3. Slide 4: With the increased use of computers in every sphere of human activity,computer scientists are faced with two crucial issues today.  Processing has to be done faster like never before  Larger or complex computation problems need to be solved
4. Slide 5: Increasing the number of transistors as per Moore’s Law isn’t a solution,as it also increases the frequency scaling and power consumption. Power consumption has been a major issue recently,as it causes a problem of processor heating. The perfect solution is PARALLELISM In hardware as well as software.
5. Slide 6: Difference With Distributed Computing When different processors/computers work on a single common goal,it is parallel computing. Eg.Ten men pulling a rope to lift up one rock,supercomputers implement parallel computing. Distributed computing is where several different computers work separately on a multi-faceted computing workload. Eg.Ten men pulling ten ropes to lift ten different rocks,employees working in an office doing their own work.
6. Slide 7: Difference With Cluster Computing A computer cluster is a group of linked computers, working together closely so that in many respects they form a single computer. Eg.,In an office of 50 employees,group of 15 doing some work,25 some other,and remaining 10 something else. Similarly,in a network of 20 computers,16 working on a common goal,whereas 4 on some other common goal. Cluster Computing is a specific case of parallel computing.
7. Slide 8: Difference With Grid Computing Grid Computing makes use of computers communicating over the Internet to work on a given problem. Eg.When 3 persons,one of them from USA,another from Japan and a third from Norway are working together online on a common project. Websites like Wikipedia,Yahoo!Answers,YouTube,FlickR or open source OS like Linux are examples of grid computing. Again,it serves a san example of parallel computing.
8. Slide 9: The Concept Of Pipelining In computing, a pipeline is a set of data processing elements connected in series, so that the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion; in that case, some amount of buffer storage is often inserted between elements.
9. Slide 10: Approaches To Parallel Computing Flynn’s Taxonomy  SISD(Single Instruction Single Data)  SIMD(Single Instruction Multiple Data)  MISD(Multiple Instruction Single Data)  MIMD(Multiple Instruction Multiple Data)
10. Slide 11: Approaches Based On Computation  Massively Parallel  Embarrassingly Parallel  Grand Challenge Problems
11. Slide 12: Massively Parallel Systems  It signifies the presence of many independent units or entire microprocessors, that run in parallel.  The term massive connotes hundreds if not thousands of such units.
12. Slide 13: Example:the Earth Simulator(Supercomputer from 2002- 2004)
13. Slide 14: Embarrassingly Parallel Systems An embarrassingly parallel system is one for which no particular effort is needed to segment the problem into a very large number of parallel tasks. Examples include surfing two websites simultaneously , or running two applications on a home computer. They lie to an end of spectrum of parallelisation where tasks can be readily parallelised.
14. Slide 15: Grand Challenge Problems A grand challenge is a fundamental problem in science or engineering, with broad applications, whose solution would be enabled by the application of high performance computing resources that could become available in the near future. Grand Challenges were USA policy terms set as goals in the late 1980s for funding high- performance computing and communications research in part in response to the Japanese 5th Generation (or Next Generation) 10-year project.
15. Slide 16: Types Of Parallelism • Bit-Level • Instructional • Data • Task
16. Slide 17: Bit-Level Parallelism When an 8-bit processor needs to add two 16- bit integers,it’s to be done in two steps.  The processor must first add the 8 lower-order bits from each integer using the standard addition instruction,  Then add the 8 higher-order bits using an add- with-carry instruction and the carry bit from the lower order addition
17. Slide 18: Instruction Level Parallelism The instructions given to a computer for processing can be divided into groups, or re- ordered and then processed without changing the final result. This is known as instruction-level parallelism. i.e.,ILP.
18. Slide 19: An Example 1. e = a + b 2. f = c + d 3. g = e * f Here, instruction 3 is dependent on instruction 1 and 2 . However,instruction 1 and 2 can be independently processed.
19. Slide 20: Data Parallelism Data parallelism focuses on distributing the data across different parallel computing nodes. It is also called as loop-level parallelism.
20. Slide 21: An Illustration In a data parallel implementation, CPU A could add all elements from the top half of the matrices, while CPU B could add all elements from the bottom half of the matrices. Since the two processors work in parallel, the job of performing matrix addition would take one half the time of performing the same operation in serial using one CPU alone.
21. Slide 22: Task Parallelism Task Parallelism focuses on distribution of tasks across different processors. It is also known as functional parallelism or control parallelism
22. Slide 23: An Example As a simple example, if we are running code on a 2-processor system (CPUs \"a\" & \"b\") in a parallel environment and we wish to do tasks \"A\" and \"B\" , it is possible to tell CPU \"a\" to do task \"A\" and CPU \"b\" to do task 'B\" simultaneously, thereby reducing the runtime of the execution.
23. Slide 24: Key Difference Between Data And Task Parallelism Data Parallelism Task Parallelism  It is the division of  It is the divisions among threads(processes) or threads(processes) or instructions or tasks instructions or tasks internally into sub-parts for themselves for execution. execution.  A task ‘A’ is divided into  A task ‘A’ and task ‘B’ sub-parts and then are processed separately processed. by different processors.
24. Slide 25: Implementation Of Parallel Computing Comput In Software When implemented in software(or rather algorithms), the terminology calls it ‘parallel programming’. An algorithm is split into pieces and then executed, as seen earlier.
25. Slide 26: Important Points In Parallel Programming Dependencies-A typical scenario when line 6 of an algorithm is dependent on lines 2,3,4 and 5 Application Checkpoints-Just like saving the algorithm, or like creating a backup point. Automatic Parallelisation-Identifying dependencies and parallelising algorithms automatically.This has achieved limited success.
26. Slide 27: Implementation Of Parallel Computing In Hardware When implemented in hardware, it is called as ‘parallel processing’. Typically,when a chunk of load for execution is divided for processing by units like cores,processors,CPUs,etc.
27. Slide 28: An Example:Intel Xeon Series Processors
28. Slide 31: References http://portal.acm.org/citation.cfm?id=290768&coll=port al&dl=ACM http://www-users.cs.umn.edu/~karypis/parbook/ www.cs.berkeley.edu/~yelick/cs267- sp04/lectures/01/lect01-intro www.cs.berkeley.edu/~demmel/cs267_Spr99/Lectures/L ect_01_1999b http://www.intel.com/technology/computing/dual- core/demo/popup/dualcore.swf www.parallel.ru/ftp/computers/intel/xeon/24896607.pdf www.intel.com
29. Slide 32: Thank You! ANY QUERIES?