A Presentation on  Parallel Computing -Ameya Waghmare(Rno 41,BE CSE) Guided by-Dr.R.P.Adgaonkar(HOD),CSE Dept.
 
<ul><li>Parallel computing is a form of computation in which many instructions are carried out  simultaneously  operating ...
<ul><li>With the increased use of computers in every sphere of human activity,computer scientists are faced with two cruci...
<ul><li>Increasing the number of transistors as per Moore’s Law isn’t a solution,as it also increases the frequency scalin...
Difference With Distributed Computing <ul><li>When different processors/computers work  on a single common goal,it is  par...
Difference With Cluster Computing <ul><li>A  computer cluster  is a group of linked computers, working together closely so...
Difference With Grid Computing <ul><li>Grid Computing   makes use of computers communicating over the Internet to work on ...
The Concept Of Pipelining <ul><li>In computing, a  pipeline  is a set of data processing elements connected in series, so ...
Approaches To Parallel Computing <ul><li>Flynn’s Taxonomy </li></ul><ul><li>SISD(Single Instruction Single Data) </li></ul...
Approaches Based On Computation <ul><li>Massively Parallel </li></ul><ul><li>Embarrassingly Parallel  </li></ul><ul><li>Gr...
Massively Parallel Systems <ul><li>It signifies the presence of many independent units or entire microprocessors, that run...
Example:the Earth Simulator(Supercomputer from 2002-2004)
Embarrassingly Parallel Systems <ul><li>An embarrassingly parallel system is one for which no particular effort is needed ...
Grand Challenge Problems <ul><li>A grand challenge is a fundamental problem in science or engineering, with broad applicat...
Types Of Parallelism <ul><li>Bit-Level </li></ul><ul><li>Instructional </li></ul><ul><li>Data </li></ul><ul><li>Task </li>...
Bit-Level Parallelism <ul><li>When an 8-bit processor needs to add two 16-bit integers,it’s to be done in two steps.  </li...
Instruction Level Parallelism <ul><li>The instructions given to a computer for processing can be divided into groups, or r...
An Example <ul><li>1. e = a + b </li></ul><ul><li>2. f = c + d </li></ul><ul><li>3. g = e * f </li></ul><ul><li>Here, inst...
Data Parallelism <ul><li>Data parallelism focuses on distributing the data across different parallel computing nodes. </li...
An Illustration <ul><li>In a data parallel implementation, CPU A could add all elements from the top half of the matrices,...
Task Parallelism <ul><li>Task Parallelism focuses on distribution of tasks across different processors. </li></ul><ul><li>...
An Example <ul><li>As a simple example, if we are running code on a 2-processor system (CPUs &quot;a&quot; & &quot;b&quot;...
Key Difference Between Data And Task Parallelism <ul><li>Data Parallelism </li></ul><ul><li>It is the division of threads(...
Implementation Of Parallel Comput ing In Software   <ul><li>When implemented in software(or rather algorithms), the termin...
Important Points In Parallel Programming <ul><li>Dependencies-A typical scenario when line 6  of an algorithm is dependent...
Implementation Of Parallel Computing In Hardware <ul><li>When implemented in hardware, it is called as ‘parallel processin...
An Example:Intel Xeon Series Processors
 
 
References <ul><li>http://portal.acm.org/citation.cfm?id=290768&coll=portal&dl=ACM </li></ul><ul><li>http://www-users.cs.u...
ANY QUERIES? <ul><li>Thank You! </li></ul>
Upcoming SlideShare
Loading in...5
×

Parallel Computing

15,473

Published on

Please contact me to download this pres.A comprehensive presentation on the field of Parallel Computing.It's applications are only growing exponentially day by days.A useful seminar covering basics,its classification and implementation thoroughly.
Visit www.ameyawaghmare.wordpress.com for more info

Published in: Technology
3 Comments
10 Likes
Statistics
Notes
No Downloads
Views
Total Views
15,473
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
995
Comments
3
Likes
10
Embeds 0
No embeds

No notes for slide

Parallel Computing

  1. 1. A Presentation on Parallel Computing -Ameya Waghmare(Rno 41,BE CSE) Guided by-Dr.R.P.Adgaonkar(HOD),CSE Dept.
  2. 3. <ul><li>Parallel computing is a form of computation in which many instructions are carried out simultaneously operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently (in parallel). </li></ul><ul><li>Why is it required? </li></ul>
  3. 4. <ul><li>With the increased use of computers in every sphere of human activity,computer scientists are faced with two crucial issues today. </li></ul><ul><li>Processing has to be done faster like never before </li></ul><ul><li>Larger or complex computation problems need to be solved </li></ul>
  4. 5. <ul><li>Increasing the number of transistors as per Moore’s Law isn’t a solution,as it also increases the frequency scaling and power consumption. </li></ul><ul><li>Power consumption has been a major issue recently,as it causes a problem of processor heating. </li></ul><ul><li>The perfect solution is PARALLELISM </li></ul><ul><li>In hardware as well as software. </li></ul>
  5. 6. Difference With Distributed Computing <ul><li>When different processors/computers work on a single common goal,it is parallel computing. </li></ul><ul><li>Eg.Ten men pulling a rope to lift up one rock,supercomputers implement parallel computing. </li></ul><ul><li>Distributed computing is where several different computers work separately on a multi-faceted computing workload. </li></ul><ul><li>Eg.Ten men pulling ten ropes to lift ten different rocks,employees working in an office doing their own work. </li></ul>
  6. 7. Difference With Cluster Computing <ul><li>A computer cluster is a group of linked computers, working together closely so that in many respects they form a single computer. </li></ul><ul><li>Eg.,In an office of 50 employees,group of 15 doing some work,25 some other,and remaining 10 something else. </li></ul><ul><li>Similarly,in a network of 20 computers,16 working on a common goal,whereas 4 on some other common goal. </li></ul><ul><li>Cluster Computing is a specific case of parallel computing. </li></ul>
  7. 8. Difference With Grid Computing <ul><li>Grid Computing makes use of computers communicating over the Internet to work on a given problem. </li></ul><ul><li>Eg.When 3 persons,one of them from USA,another from Japan and a third from Norway are working together online on a common project. </li></ul><ul><li>Websites like Wikipedia,Yahoo!Answers,YouTube,FlickR or open source OS like Linux are examples of grid computing. </li></ul><ul><li>Again,it serves a san example of parallel computing. </li></ul>
  8. 9. The Concept Of Pipelining <ul><li>In computing, a pipeline is a set of data processing elements connected in series, so that the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion; in that case, some amount of buffer storage is often inserted between elements. </li></ul>
  9. 10. Approaches To Parallel Computing <ul><li>Flynn’s Taxonomy </li></ul><ul><li>SISD(Single Instruction Single Data) </li></ul><ul><li>SIMD(Single Instruction Multiple Data) </li></ul><ul><li>MISD(Multiple Instruction Single Data) </li></ul><ul><li>MIMD(Multiple Instruction Multiple Data) </li></ul>
  10. 11. Approaches Based On Computation <ul><li>Massively Parallel </li></ul><ul><li>Embarrassingly Parallel </li></ul><ul><li>Grand Challenge Problems </li></ul>
  11. 12. Massively Parallel Systems <ul><li>It signifies the presence of many independent units or entire microprocessors, that run in parallel. </li></ul><ul><li>The term massive connotes hundreds if not thousands of such units. </li></ul>
  12. 13. Example:the Earth Simulator(Supercomputer from 2002-2004)
  13. 14. Embarrassingly Parallel Systems <ul><li>An embarrassingly parallel system is one for which no particular effort is needed to segment the problem into a very large number of parallel tasks. </li></ul><ul><li>Examples include surfing two websites simultaneously , or running two applications on a home computer. </li></ul><ul><li>They lie to an end of spectrum of parallelisation where tasks can be readily parallelised. </li></ul>
  14. 15. Grand Challenge Problems <ul><li>A grand challenge is a fundamental problem in science or engineering, with broad applications, whose solution would be enabled by the application of high performance computing resources that could become available in the near future. </li></ul><ul><li>Grand Challenges were USA policy terms set as goals in the late 1980s for funding high-performance computing and communications research in part in response to the Japanese 5th Generation (or Next Generation) 10-year project. </li></ul>
  15. 16. Types Of Parallelism <ul><li>Bit-Level </li></ul><ul><li>Instructional </li></ul><ul><li>Data </li></ul><ul><li>Task </li></ul>
  16. 17. Bit-Level Parallelism <ul><li>When an 8-bit processor needs to add two 16-bit integers,it’s to be done in two steps. </li></ul><ul><li>The processor must first add the 8 lower-order bits from each integer using the standard addition instruction, </li></ul><ul><li>Then add the 8 higher-order bits using an add-with-carry instruction and the carry bit from the lower order addition </li></ul>
  17. 18. Instruction Level Parallelism <ul><li>The instructions given to a computer for processing can be divided into groups, or re-ordered and then processed without changing the final result. </li></ul><ul><li>This is known as instruction-level parallelism. </li></ul><ul><li>i.e.,ILP. </li></ul>
  18. 19. An Example <ul><li>1. e = a + b </li></ul><ul><li>2. f = c + d </li></ul><ul><li>3. g = e * f </li></ul><ul><li>Here, instruction 3 is dependent on instruction 1 and 2 . </li></ul><ul><li>However,instruction 1 and 2 can be independently processed. </li></ul>
  19. 20. Data Parallelism <ul><li>Data parallelism focuses on distributing the data across different parallel computing nodes. </li></ul><ul><li>It is also called as loop-level parallelism. </li></ul>
  20. 21. An Illustration <ul><li>In a data parallel implementation, CPU A could add all elements from the top half of the matrices, while CPU B could add all elements from the bottom half of the matrices. </li></ul><ul><li>Since the two processors work in parallel, the job of performing matrix addition would take one half the time of performing the same operation in serial using one CPU alone. </li></ul>
  21. 22. Task Parallelism <ul><li>Task Parallelism focuses on distribution of tasks across different processors. </li></ul><ul><li>It is also known as functional parallelism or control parallelism </li></ul>
  22. 23. An Example <ul><li>As a simple example, if we are running code on a 2-processor system (CPUs &quot;a&quot; & &quot;b&quot;) in a parallel environment and we wish to do tasks &quot;A&quot; and &quot;B&quot; , it is possible to tell CPU &quot;a&quot; to do task &quot;A&quot; and CPU &quot;b&quot; to do task 'B&quot; simultaneously, thereby reducing the runtime of the execution. </li></ul>
  23. 24. Key Difference Between Data And Task Parallelism <ul><li>Data Parallelism </li></ul><ul><li>It is the division of threads(processes) or instructions or tasks internally into sub-parts for execution. </li></ul><ul><li>A task ‘A’ is divided into sub-parts and then processed. </li></ul><ul><li>Task Parallelism </li></ul><ul><li>It is the divisions among threads(processes) or instructions or tasks themselves for execution. </li></ul><ul><li>A task ‘A’ and task ‘B’ are processed separately by different processors. </li></ul>
  24. 25. Implementation Of Parallel Comput ing In Software <ul><li>When implemented in software(or rather algorithms), the terminology calls it ‘parallel programming’. </li></ul><ul><li>An algorithm is split into pieces and then executed, as seen earlier. </li></ul>
  25. 26. Important Points In Parallel Programming <ul><li>Dependencies-A typical scenario when line 6 of an algorithm is dependent on lines 2,3,4 and 5 </li></ul><ul><li>Application Checkpoints-Just like saving the algorithm, or like creating a backup point. </li></ul><ul><li>Automatic Parallelisation-Identifying dependencies and parallelising algorithms automatically.This has achieved limited success. </li></ul>
  26. 27. Implementation Of Parallel Computing In Hardware <ul><li>When implemented in hardware, it is called as ‘parallel processing’. </li></ul><ul><li>Typically,when a chunk of load for execution is divided for processing by units like cores,processors,CPUs,etc. </li></ul>
  27. 28. An Example:Intel Xeon Series Processors
  28. 31. References <ul><li>http://portal.acm.org/citation.cfm?id=290768&coll=portal&dl=ACM </li></ul><ul><li>http://www-users.cs.umn.edu/~karypis/parbook/ </li></ul><ul><li>www.cs.berkeley.edu/~yelick/cs267-sp04/lectures/01/lect01-intro </li></ul><ul><li>www.cs.berkeley.edu/~demmel/cs267_Spr99/Lectures/Lect_01_1999b </li></ul><ul><li>http://www.intel.com/technology/computing/dual-core/demo/popup/dualcore.swf </li></ul><ul><li>www.parallel.ru/ftp/computers/intel/xeon/24896607.pdf </li></ul><ul><li>www.intel.com </li></ul>
  29. 32. ANY QUERIES? <ul><li>Thank You! </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×