SlideShare a Scribd company logo
Create an Office Mix 
Let’s Get Started…
Topic Overview 
•What is Parallel Computing? 
•Why Parallel Computing ? 
•Scope of Parallel Computing Applications
What is Parallel Computing?
What is Parallel Computing? 
• Traditionally, software has been written for serial 
computation: 
– To be run on a single computer having a single 
Central Processing Unit (CPU); 
– A problem is broken into a discrete series of 
instructions. 
– Instructions are executed one after another. 
– Only one instruction may execute at any moment 
in time.
• In the simplest sense, parallel computing is the 
simultaneous use of multiple compute resources to solve a 
computational problem. 
– To be run using multiple CPUs 
– A problem is broken into discrete parts that can be 
solved concurrently 
– Each part is further broken down to a series of 
instructions 
• Instructions from each part execute simultaneously on 
different CPUs
Why parallel computing ?
Limitations of Serial Computing….. 
•Limits to serial computing - both physical and practical reasons pose 
significant constraints to simply building ever faster serial computers. 
•Transmission speeds - the speed of a serial computer is directly dependent 
upon how fast data can move through hardware. Absolute limits are the 
speed of light (30 cm/nanosecond) and the transmission limit of copper wire 
(9 cm/nanosecond). Increasing speeds necessitate increasing proximity of 
processing elements. 
•Limits to miniaturization - processor technology is allowing an increasing 
number of transistors to be placed on a chip. However, even with molecular or 
atomic-level components, a limit will be reached on how small components 
can be. 
•Economic limitations - it is increasingly expensive to make a single processor 
faster. Using a larger number of moderately fast commodity processors to 
achieve the same (or better) performance is less expensive.
Why Parallel 
Computing? 
The primary reasons for using 
parallel computing: 
•Save time - wall clock time 
•Solve larger problems 
•Provide concurrency (do multiple 
things at the same time)
Other reasons might include: 
– Taking advantage of non-local resources - using 
available compute resources on a wide area network, or 
even the Internet when local compute resources are 
scarce. 
– Cost savings - using multiple "cheap" computing 
resources instead of paying for time on a 
supercomputer. 
– Overcoming memory constraints - single computers 
have very finite memory resources. For large problems, 
using the memories of multiple computers may 
overcome this obstacle.
Moore’s Law Moore's law states [1965]: 
``The complexity for minimum 
component costs has increased at a rate of 
roughly a factor of two per year. Certainly 
over the short term this rate can be expected 
to continue, if not to increase. Over the 
longer term, the rate of increase is a bit more 
uncertain, although there is no reason to 
believe it will not remain nearly constant for 
at least 10 years. That means by 1975, the 
number of components per integrated circuit 
for minimum cost will be 65,000.''
Moore attributed this doubling rate to exponential behavior 
of die sizes, finer minimum dimensions, and ``circuit and 
device cleverness''. 
In 1975, he revised this law as follows: 
``There is no room left to squeeze anything out by 
being clever. Going forward from here we have to depend on the 
two size factors - bigger dies and finer dimensions.'' 
He revised his rate of circuit complexity doubling to 
18 months and projected from 1975 onwards at this reduced 
rate.
The Computational Power Argument 
•If one is to buy into Moore's law, the question still 
remains - how does one translate transistors into useful 
OPS (operations per second)? 
•The logical recourse is to rely on parallelism, both 
implicit and explicit. 
•Most serial (or seemingly serial) processors rely 
extensively on implicit parallelism. 
•We focus in this class, for the most part, on explicit 
parallelism.
The Memory/Disk Speed Argument 
• While clock rates of high-end processors have increased at 
roughly 40% per year over the past decade, DRAM access times 
have only improved at the rate of roughly 10% per year over this 
interval. 
• This mismatch in speeds causes significant performance 
bottlenecks. 
• Parallel platforms provide increased bandwidth to the memory 
system. 
• Parallel platforms also provide higher aggregate caches. 
• Principles of locality of data reference and bulk access, which 
guide parallel algorithm design also apply to memory 
optimization. 
• Some of the fastest growing applications of parallel computing 
utilize not their raw computational speed, rather their ability to 
pump data to memory and disk faster.
The Data Communication Argument 
• As the network evolves, the vision of the Internet as 
one large computing platform has emerged. 
• This view is exploited by applications such as 
SETI@home and Folding@home. 
• In many other applications (typically databases and 
data mining) the volume of data is such that they 
cannot be moved. 
• Any analyses on this data must be performed over 
the network using parallel techniques.
Scope of Parallel 
Computing Applications
Parallelism finds applications in very diverse 
application domains for different motivating reasons. 
These range from improved application performance to 
cost considerations.
• Design of airfoils (optimizing lift, drag, 
stability), internal combustion engines 
(optimizing charge distribution, burn), 
high-speed circuits (layouts for delays and 
capacitive and inductive effects), and 
structures (optimizing structural integrity, 
design parameters, cost, etc.). 
• Design and simulation of micro- and 
nano-scale systems. 
• Process optimization, operations research. 
Applications in 
Engineering 
and Design
• Functional and structural characterization of 
genes and proteins. 
• Advances in computational physics and chemistry 
have explored new materials, understanding of 
chemical pathways, and more efficient processes. 
• Applications in astrophysics have explored the 
evolution of galaxies, thermonuclear processes, 
and the analysis of extremely large datasets from 
telescopes. 
• Weather modeling, mineral prospecting, flood 
prediction, etc., are other important applications. 
• Bioinformatics and astrophysics also present 
some of the most challenging problems with 
respect to analyzing extremely large datasets. 
Scientific 
Applications
•Some of the largest parallel computers 
power the wall street! 
•Data mining and analysis for optimizing 
business and marketing decisions. 
•Large scale servers (mail and web servers) 
are often implemented using parallel 
platforms. 
•Applications such as information retrieval 
and search are typically powered by large 
clusters. 
Commercial 
Applications
• Network intrusion detection, cryptography, 
multiparty computations are some of the 
core users of parallel computing 
techniques. 
• Embedded systems increasingly rely on 
distributed control algorithms. 
• A modern automobile consists of tens of 
processors communicating to perform 
complex tasks for optimizing handling and 
performance. 
• Conventional structured peer-to-peer 
networks impose overlay networks and 
utilize algorithms directly from parallel 
computing. 
Applications 
in Computer 
Systems
Thank You!

More Related Content

What's hot

Process management in os
Process management in osProcess management in os
Process management in os
Miong Lazaro
 
Processes and Processors in Distributed Systems
Processes and Processors in Distributed SystemsProcesses and Processors in Distributed Systems
Processes and Processors in Distributed Systems
Dr Sandeep Kumar Poonia
 
chap 18 multicore computers
chap 18 multicore computers chap 18 multicore computers
chap 18 multicore computers
Sher Shah Merkhel
 
Distributed Systems Real Life Applications
Distributed Systems Real Life ApplicationsDistributed Systems Real Life Applications
Distributed Systems Real Life Applications
Aman Srivastava
 
Memory management
Memory managementMemory management
Memory management
Muhammad Fayyaz
 
Introduction To Parallel Computing
Introduction To Parallel ComputingIntroduction To Parallel Computing
Introduction To Parallel Computing
Jörn Dinkla
 
Hardware multithreading
Hardware multithreadingHardware multithreading
Hardware multithreading
Fraboni Ec
 
Sequential consistency model
Sequential consistency modelSequential consistency model
Sequential consistency model
Bharathi Lakshmi Pon
 
Foult Tolerence In Distributed System
Foult Tolerence In Distributed SystemFoult Tolerence In Distributed System
Foult Tolerence In Distributed System
Rajan Kumar
 
Distributed Operating System_1
Distributed Operating System_1Distributed Operating System_1
Distributed Operating System_1
Dr Sandeep Kumar Poonia
 
Introduction to Distributed System
Introduction to Distributed SystemIntroduction to Distributed System
Introduction to Distributed System
Sunita Sahu
 
High Performance Computing
High Performance ComputingHigh Performance Computing
High Performance Computing
Divyen Patel
 
Lecture 3 parallel programming platforms
Lecture 3   parallel programming platformsLecture 3   parallel programming platforms
Lecture 3 parallel programming platforms
Vajira Thambawita
 
High performance computing
High performance computingHigh performance computing
High performance computing
punjab engineering college, chandigarh
 
Physical organization of parallel platforms
Physical organization of parallel platformsPhysical organization of parallel platforms
Physical organization of parallel platforms
Syed Zaid Irshad
 
Limitations of memory system performance
Limitations of memory system performanceLimitations of memory system performance
Limitations of memory system performance
Syed Zaid Irshad
 
Concurrency
ConcurrencyConcurrency
Concurrency
rizwanaabassi
 
Multiprocessor architecture
Multiprocessor architectureMultiprocessor architecture
Multiprocessor architecture
Arpan Baishya
 
Models of Distributed System
Models of Distributed SystemModels of Distributed System
Models of Distributed System
Ashish KC
 
Distributed Systems Introduction and Importance
Distributed Systems Introduction and Importance Distributed Systems Introduction and Importance
Distributed Systems Introduction and Importance
SHIKHA GAUTAM
 

What's hot (20)

Process management in os
Process management in osProcess management in os
Process management in os
 
Processes and Processors in Distributed Systems
Processes and Processors in Distributed SystemsProcesses and Processors in Distributed Systems
Processes and Processors in Distributed Systems
 
chap 18 multicore computers
chap 18 multicore computers chap 18 multicore computers
chap 18 multicore computers
 
Distributed Systems Real Life Applications
Distributed Systems Real Life ApplicationsDistributed Systems Real Life Applications
Distributed Systems Real Life Applications
 
Memory management
Memory managementMemory management
Memory management
 
Introduction To Parallel Computing
Introduction To Parallel ComputingIntroduction To Parallel Computing
Introduction To Parallel Computing
 
Hardware multithreading
Hardware multithreadingHardware multithreading
Hardware multithreading
 
Sequential consistency model
Sequential consistency modelSequential consistency model
Sequential consistency model
 
Foult Tolerence In Distributed System
Foult Tolerence In Distributed SystemFoult Tolerence In Distributed System
Foult Tolerence In Distributed System
 
Distributed Operating System_1
Distributed Operating System_1Distributed Operating System_1
Distributed Operating System_1
 
Introduction to Distributed System
Introduction to Distributed SystemIntroduction to Distributed System
Introduction to Distributed System
 
High Performance Computing
High Performance ComputingHigh Performance Computing
High Performance Computing
 
Lecture 3 parallel programming platforms
Lecture 3   parallel programming platformsLecture 3   parallel programming platforms
Lecture 3 parallel programming platforms
 
High performance computing
High performance computingHigh performance computing
High performance computing
 
Physical organization of parallel platforms
Physical organization of parallel platformsPhysical organization of parallel platforms
Physical organization of parallel platforms
 
Limitations of memory system performance
Limitations of memory system performanceLimitations of memory system performance
Limitations of memory system performance
 
Concurrency
ConcurrencyConcurrency
Concurrency
 
Multiprocessor architecture
Multiprocessor architectureMultiprocessor architecture
Multiprocessor architecture
 
Models of Distributed System
Models of Distributed SystemModels of Distributed System
Models of Distributed System
 
Distributed Systems Introduction and Importance
Distributed Systems Introduction and Importance Distributed Systems Introduction and Importance
Distributed Systems Introduction and Importance
 

Similar to Introduction to Parallel Computing

Chap1 slides
Chap1 slidesChap1 slides
Chap1 slides
BaliThorat1
 
High–Performance Computing
High–Performance ComputingHigh–Performance Computing
High–Performance Computing
BRAC University Computer Club
 
3 - Grid Computing.pptx
3 - Grid Computing.pptx3 - Grid Computing.pptx
3 - Grid Computing.pptx
RiazSalim1
 
CC unit 1.pptx
CC unit 1.pptxCC unit 1.pptx
CC unit 1.pptx
DivyaRadharapu1
 
42 grid computing
42 grid computing42 grid computing
42 grid computing
SALMAN SHAIKH
 
42 grid computing
42 grid computing42 grid computing
42 grid computing
SALMAN SHAIKH
 
Grid computing
Grid computingGrid computing
Cluster Computing
Cluster ComputingCluster Computing
Cluster Computing
Hitesh Mohapatra
 
Fundamentals.pptx
Fundamentals.pptxFundamentals.pptx
Fundamentals.pptx
dhivyak49
 
Simulation of Heterogeneous Cloud Infrastructures
Simulation of Heterogeneous Cloud InfrastructuresSimulation of Heterogeneous Cloud Infrastructures
Simulation of Heterogeneous Cloud Infrastructures
CloudLightning
 
Chap2 slides
Chap2 slidesChap2 slides
Chap2 slides
BaliThorat1
 
CF-Chapitre1.pdf
CF-Chapitre1.pdfCF-Chapitre1.pdf
CF-Chapitre1.pdf
mohameddouhabi2002
 
Grid Computing
Grid ComputingGrid Computing
Grid Computing
Phương Đông
 
Chap 2 classification of parralel architecture and introduction to parllel p...
Chap 2  classification of parralel architecture and introduction to parllel p...Chap 2  classification of parralel architecture and introduction to parllel p...
Chap 2 classification of parralel architecture and introduction to parllel p...
Malobe Lottin Cyrille Marcel
 
CC LECTURE NOTES (1).pdf
CC LECTURE NOTES (1).pdfCC LECTURE NOTES (1).pdf
CC LECTURE NOTES (1).pdf
HasanAfwaaz1
 
Grid computing Seminar PPT
Grid computing Seminar PPTGrid computing Seminar PPT
Grid computing Seminar PPT
Upender Upr
 
Aca module 1
Aca module 1Aca module 1
Aca module 1
Avinash_N Rao
 
Cloud Computing and Virtualization Overview by Amr Ali
Cloud Computing and Virtualization Overview by Amr AliCloud Computing and Virtualization Overview by Amr Ali
Cloud Computing and Virtualization Overview by Amr Ali
Amr Ali
 
(19-23)CC Unit-1 ppt.pptx
(19-23)CC Unit-1 ppt.pptx(19-23)CC Unit-1 ppt.pptx
(19-23)CC Unit-1 ppt.pptx
NithishaYadavv
 
54665962-Nav-Cluster-Computing.pptx
54665962-Nav-Cluster-Computing.pptx54665962-Nav-Cluster-Computing.pptx
54665962-Nav-Cluster-Computing.pptx
YashAhire28
 

Similar to Introduction to Parallel Computing (20)

Chap1 slides
Chap1 slidesChap1 slides
Chap1 slides
 
High–Performance Computing
High–Performance ComputingHigh–Performance Computing
High–Performance Computing
 
3 - Grid Computing.pptx
3 - Grid Computing.pptx3 - Grid Computing.pptx
3 - Grid Computing.pptx
 
CC unit 1.pptx
CC unit 1.pptxCC unit 1.pptx
CC unit 1.pptx
 
42 grid computing
42 grid computing42 grid computing
42 grid computing
 
42 grid computing
42 grid computing42 grid computing
42 grid computing
 
Grid computing
Grid computingGrid computing
Grid computing
 
Cluster Computing
Cluster ComputingCluster Computing
Cluster Computing
 
Fundamentals.pptx
Fundamentals.pptxFundamentals.pptx
Fundamentals.pptx
 
Simulation of Heterogeneous Cloud Infrastructures
Simulation of Heterogeneous Cloud InfrastructuresSimulation of Heterogeneous Cloud Infrastructures
Simulation of Heterogeneous Cloud Infrastructures
 
Chap2 slides
Chap2 slidesChap2 slides
Chap2 slides
 
CF-Chapitre1.pdf
CF-Chapitre1.pdfCF-Chapitre1.pdf
CF-Chapitre1.pdf
 
Grid Computing
Grid ComputingGrid Computing
Grid Computing
 
Chap 2 classification of parralel architecture and introduction to parllel p...
Chap 2  classification of parralel architecture and introduction to parllel p...Chap 2  classification of parralel architecture and introduction to parllel p...
Chap 2 classification of parralel architecture and introduction to parllel p...
 
CC LECTURE NOTES (1).pdf
CC LECTURE NOTES (1).pdfCC LECTURE NOTES (1).pdf
CC LECTURE NOTES (1).pdf
 
Grid computing Seminar PPT
Grid computing Seminar PPTGrid computing Seminar PPT
Grid computing Seminar PPT
 
Aca module 1
Aca module 1Aca module 1
Aca module 1
 
Cloud Computing and Virtualization Overview by Amr Ali
Cloud Computing and Virtualization Overview by Amr AliCloud Computing and Virtualization Overview by Amr Ali
Cloud Computing and Virtualization Overview by Amr Ali
 
(19-23)CC Unit-1 ppt.pptx
(19-23)CC Unit-1 ppt.pptx(19-23)CC Unit-1 ppt.pptx
(19-23)CC Unit-1 ppt.pptx
 
54665962-Nav-Cluster-Computing.pptx
54665962-Nav-Cluster-Computing.pptx54665962-Nav-Cluster-Computing.pptx
54665962-Nav-Cluster-Computing.pptx
 

More from Roshan Karunarathna

UI/UX presentation by Roshan Karunarathna
UI/UX presentation by Roshan KarunarathnaUI/UX presentation by Roshan Karunarathna
UI/UX presentation by Roshan Karunarathna
Roshan Karunarathna
 
LIVE CHAT ETIQUETTE
LIVE CHAT ETIQUETTELIVE CHAT ETIQUETTE
LIVE CHAT ETIQUETTE
Roshan Karunarathna
 
Iplanet
IplanetIplanet
onlinemarketing
onlinemarketingonlinemarketing
onlinemarketing
Roshan Karunarathna
 
Pay Pal Introduction.........!
Pay Pal Introduction.........!Pay Pal Introduction.........!
Pay Pal Introduction.........!
Roshan Karunarathna
 
Chou fasman algorithm for protein structure prediction
Chou fasman algorithm for protein structure predictionChou fasman algorithm for protein structure prediction
Chou fasman algorithm for protein structure prediction
Roshan Karunarathna
 
Iterative and Incremental Development (RAD)
Iterative  and Incremental Development (RAD)Iterative  and Incremental Development (RAD)
Iterative and Incremental Development (RAD)
Roshan Karunarathna
 
Introduction to backwards learning algorithm
Introduction to backwards learning algorithmIntroduction to backwards learning algorithm
Introduction to backwards learning algorithm
Roshan Karunarathna
 

More from Roshan Karunarathna (8)

UI/UX presentation by Roshan Karunarathna
UI/UX presentation by Roshan KarunarathnaUI/UX presentation by Roshan Karunarathna
UI/UX presentation by Roshan Karunarathna
 
LIVE CHAT ETIQUETTE
LIVE CHAT ETIQUETTELIVE CHAT ETIQUETTE
LIVE CHAT ETIQUETTE
 
Iplanet
IplanetIplanet
Iplanet
 
onlinemarketing
onlinemarketingonlinemarketing
onlinemarketing
 
Pay Pal Introduction.........!
Pay Pal Introduction.........!Pay Pal Introduction.........!
Pay Pal Introduction.........!
 
Chou fasman algorithm for protein structure prediction
Chou fasman algorithm for protein structure predictionChou fasman algorithm for protein structure prediction
Chou fasman algorithm for protein structure prediction
 
Iterative and Incremental Development (RAD)
Iterative  and Incremental Development (RAD)Iterative  and Incremental Development (RAD)
Iterative and Incremental Development (RAD)
 
Introduction to backwards learning algorithm
Introduction to backwards learning algorithmIntroduction to backwards learning algorithm
Introduction to backwards learning algorithm
 

Recently uploaded

Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 

Recently uploaded (20)

Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 

Introduction to Parallel Computing

  • 1. Create an Office Mix Let’s Get Started…
  • 2. Topic Overview •What is Parallel Computing? •Why Parallel Computing ? •Scope of Parallel Computing Applications
  • 3. What is Parallel Computing?
  • 4. What is Parallel Computing? • Traditionally, software has been written for serial computation: – To be run on a single computer having a single Central Processing Unit (CPU); – A problem is broken into a discrete series of instructions. – Instructions are executed one after another. – Only one instruction may execute at any moment in time.
  • 5. • In the simplest sense, parallel computing is the simultaneous use of multiple compute resources to solve a computational problem. – To be run using multiple CPUs – A problem is broken into discrete parts that can be solved concurrently – Each part is further broken down to a series of instructions • Instructions from each part execute simultaneously on different CPUs
  • 7. Limitations of Serial Computing….. •Limits to serial computing - both physical and practical reasons pose significant constraints to simply building ever faster serial computers. •Transmission speeds - the speed of a serial computer is directly dependent upon how fast data can move through hardware. Absolute limits are the speed of light (30 cm/nanosecond) and the transmission limit of copper wire (9 cm/nanosecond). Increasing speeds necessitate increasing proximity of processing elements. •Limits to miniaturization - processor technology is allowing an increasing number of transistors to be placed on a chip. However, even with molecular or atomic-level components, a limit will be reached on how small components can be. •Economic limitations - it is increasingly expensive to make a single processor faster. Using a larger number of moderately fast commodity processors to achieve the same (or better) performance is less expensive.
  • 8. Why Parallel Computing? The primary reasons for using parallel computing: •Save time - wall clock time •Solve larger problems •Provide concurrency (do multiple things at the same time)
  • 9. Other reasons might include: – Taking advantage of non-local resources - using available compute resources on a wide area network, or even the Internet when local compute resources are scarce. – Cost savings - using multiple "cheap" computing resources instead of paying for time on a supercomputer. – Overcoming memory constraints - single computers have very finite memory resources. For large problems, using the memories of multiple computers may overcome this obstacle.
  • 10. Moore’s Law Moore's law states [1965]: ``The complexity for minimum component costs has increased at a rate of roughly a factor of two per year. Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years. That means by 1975, the number of components per integrated circuit for minimum cost will be 65,000.''
  • 11. Moore attributed this doubling rate to exponential behavior of die sizes, finer minimum dimensions, and ``circuit and device cleverness''. In 1975, he revised this law as follows: ``There is no room left to squeeze anything out by being clever. Going forward from here we have to depend on the two size factors - bigger dies and finer dimensions.'' He revised his rate of circuit complexity doubling to 18 months and projected from 1975 onwards at this reduced rate.
  • 12. The Computational Power Argument •If one is to buy into Moore's law, the question still remains - how does one translate transistors into useful OPS (operations per second)? •The logical recourse is to rely on parallelism, both implicit and explicit. •Most serial (or seemingly serial) processors rely extensively on implicit parallelism. •We focus in this class, for the most part, on explicit parallelism.
  • 13. The Memory/Disk Speed Argument • While clock rates of high-end processors have increased at roughly 40% per year over the past decade, DRAM access times have only improved at the rate of roughly 10% per year over this interval. • This mismatch in speeds causes significant performance bottlenecks. • Parallel platforms provide increased bandwidth to the memory system. • Parallel platforms also provide higher aggregate caches. • Principles of locality of data reference and bulk access, which guide parallel algorithm design also apply to memory optimization. • Some of the fastest growing applications of parallel computing utilize not their raw computational speed, rather their ability to pump data to memory and disk faster.
  • 14. The Data Communication Argument • As the network evolves, the vision of the Internet as one large computing platform has emerged. • This view is exploited by applications such as SETI@home and Folding@home. • In many other applications (typically databases and data mining) the volume of data is such that they cannot be moved. • Any analyses on this data must be performed over the network using parallel techniques.
  • 15. Scope of Parallel Computing Applications
  • 16. Parallelism finds applications in very diverse application domains for different motivating reasons. These range from improved application performance to cost considerations.
  • 17. • Design of airfoils (optimizing lift, drag, stability), internal combustion engines (optimizing charge distribution, burn), high-speed circuits (layouts for delays and capacitive and inductive effects), and structures (optimizing structural integrity, design parameters, cost, etc.). • Design and simulation of micro- and nano-scale systems. • Process optimization, operations research. Applications in Engineering and Design
  • 18. • Functional and structural characterization of genes and proteins. • Advances in computational physics and chemistry have explored new materials, understanding of chemical pathways, and more efficient processes. • Applications in astrophysics have explored the evolution of galaxies, thermonuclear processes, and the analysis of extremely large datasets from telescopes. • Weather modeling, mineral prospecting, flood prediction, etc., are other important applications. • Bioinformatics and astrophysics also present some of the most challenging problems with respect to analyzing extremely large datasets. Scientific Applications
  • 19. •Some of the largest parallel computers power the wall street! •Data mining and analysis for optimizing business and marketing decisions. •Large scale servers (mail and web servers) are often implemented using parallel platforms. •Applications such as information retrieval and search are typically powered by large clusters. Commercial Applications
  • 20. • Network intrusion detection, cryptography, multiparty computations are some of the core users of parallel computing techniques. • Embedded systems increasingly rely on distributed control algorithms. • A modern automobile consists of tens of processors communicating to perform complex tasks for optimizing handling and performance. • Conventional structured peer-to-peer networks impose overlay networks and utilize algorithms directly from parallel computing. Applications in Computer Systems