From the perspective of Design and Analysis of Algorithm. I made these slide by collecting data from many sites.
I am Danish Javed. Student of BSCS Hons. at ITU Information Technology University Lahore, Punjab, Pakistan.
Parallel computing is a type of computation in which many calculations or the execution of processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. There are several different forms of parallel computing: bit-level, instruction-level, data, and task parallelism. Parallelism has been employed for many years, mainly in high-performance computing, but interest in it has grown lately due to the physical constraints preventing frequency scaling. As power consumption (and consequently heat generation) by computers has become a concern in recent years, parallel computing has become the dominant paradigm in computer architecture, mainly in the form of multi-core processors.
Parallel computing is a type of computation in which many calculations or the execution of processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. There are several different forms of parallel computing: bit-level, instruction-level, data, and task parallelism. Parallelism has been employed for many years, mainly in high-performance computing, but interest in it has grown lately due to the physical constraints preventing frequency scaling. As power consumption (and consequently heat generation) by computers has become a concern in recent years, parallel computing has become the dominant paradigm in computer architecture, mainly in the form of multi-core processors.
This chapter discusses various classification attributed to parallel architectures. It also introduces related parallel programming models and presents the actions of these models on parallel architectures. Notions such as Data parallelism Task parallelism, Tighty and Coupled system, UMA/NUMA, Multicore computing, Symmetric multiprocessing, Distributed Computing, Cluster computing, Shared memory without thread/Thread, etc..
This slides contains assymptotic notations, recurrence relation like subtitution method, iteration method, master method and recursion tree method and sorting algorithms like merge sort, quick sort, heap sort, counting sort, radix sort and bucket sort.
An explicitly parallel program must specify concurrency and interaction between concurrent subtasks.
The former is sometimes also referred to as the control structure and the latter as the communication model.
INTRODUCTIONTO OPERATING SYSTEM
What is an Operating System?
Mainframe Systems
Desktop Systems
Multiprocessor Systems
Distributed Systems
Clustered System
Real -Time Systems
Handheld Systems
Computing Environments
This chapter discusses various classification attributed to parallel architectures. It also introduces related parallel programming models and presents the actions of these models on parallel architectures. Notions such as Data parallelism Task parallelism, Tighty and Coupled system, UMA/NUMA, Multicore computing, Symmetric multiprocessing, Distributed Computing, Cluster computing, Shared memory without thread/Thread, etc..
This slides contains assymptotic notations, recurrence relation like subtitution method, iteration method, master method and recursion tree method and sorting algorithms like merge sort, quick sort, heap sort, counting sort, radix sort and bucket sort.
An explicitly parallel program must specify concurrency and interaction between concurrent subtasks.
The former is sometimes also referred to as the control structure and the latter as the communication model.
INTRODUCTIONTO OPERATING SYSTEM
What is an Operating System?
Mainframe Systems
Desktop Systems
Multiprocessor Systems
Distributed Systems
Clustered System
Real -Time Systems
Handheld Systems
Computing Environments
These slides are about parallel sorting algorithms. In which four types of sorting algorithms are discussed with the comparison between their sequential and parallel ways. The four algorithms which are included are: Bubble sort, merge sort, Bitonic sort and Shear sort.
An OpenCL Method of Parallel Sorting Algorithms for GPU ArchitectureWaqas Tariq
In this paper, we present a comparative performance analysis of different parallel sorting algorithms: Bitonic sort and Parallel Radix Sort. In order to study the interaction between the algorithms and architecture, we implemented both the algorithms in OpenCL and compared its performance with Quick Sort algorithm, the fastest algorithm. In our simulation, we have used Intel Core2Duo CPU 2.67GHz and NVidia Quadro FX 3800 as graphical processing unit.
Here, we look at the problem of multi-threading algorithms and its implication for the creation of parallel algorithms. However, this is a barely scant introduction to a subject that requires an entire class. Nevertheless, in the slides we look at the matrix multiplication and quick sort implementations.
This describes a new parallel sorting algorithm
suitable for a variety of multiprocessor architectures.
Parallel Sorting by Regular Sampling (PSRS) finds pivots
for partitioning the data into ordered subsets of approximately equal size by using a regular sample from
sorted sublists of the data.
Redesigning a large B2B website - The FusionCharts revamping storyFusionCharts
A detailed look at everything that went behind the redesign of the FusionCharts website - objectives, tech stack and server hardware, information architecture, front-end decisions to make it responsive, design tradeoffs, SEO, and analytics. The decisions we made, the process we followed, the learnings we had and the final results.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
The International Journal of Engineering and Science (The IJES)theijes
The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.
GRAPH MATCHING ALGORITHM FOR TASK ASSIGNMENT PROBLEMIJCSEA Journal
Task assignment is one of the most challenging problems in distributed computing environment. An optimal task assignment guarantees minimum turnaround time for a given architecture. Several approaches of optimal task assignment have been proposed by various researchers ranging from graph partitioning based tools to heuristic graph matching. Using heuristic graph matching, it is often impossible to get optimal task assignment for practical test cases within an acceptable time limit. In this paper, we have parallelized the basic heuristic graph-matching algorithm of task assignment which is suitable only for cases where processors and inter processor links are homogeneous. This proposal is a derivative of the basic task assignment methodology using heuristic graph matching. The results show that near optimal assignments are obtained much faster than the sequential program in all the cases with reasonable speed-up.
Data Parallel and Object Oriented ModelNikhil Sharma
All the content is taken from Advance Computer Architecture book. Which (10.1.3 and 10.1.4)
This PPT covers the basics of Data-Parallel Model and Object-Oriented Model.
Please contact me to download this pres.A comprehensive presentation on the field of Parallel Computing.It's applications are only growing exponentially day by days.A useful seminar covering basics,its classification and implementation thoroughly.
Visit www.ameyawaghmare.wordpress.com for more info
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
The Art Pastor's Guide to Sabbath | Steve ThomasonSteve Thomason
What is the purpose of the Sabbath Law in the Torah. It is interesting to compare how the context of the law shifts from Exodus to Deuteronomy. Who gets to rest, and why?
The Roman Empire A Historical Colossus.pdfkaushalkr1407
The Roman Empire, a vast and enduring power, stands as one of history's most remarkable civilizations, leaving an indelible imprint on the world. It emerged from the Roman Republic, transitioning into an imperial powerhouse under the leadership of Augustus Caesar in 27 BCE. This transformation marked the beginning of an era defined by unprecedented territorial expansion, architectural marvels, and profound cultural influence.
The empire's roots lie in the city of Rome, founded, according to legend, by Romulus in 753 BCE. Over centuries, Rome evolved from a small settlement to a formidable republic, characterized by a complex political system with elected officials and checks on power. However, internal strife, class conflicts, and military ambitions paved the way for the end of the Republic. Julius Caesar’s dictatorship and subsequent assassination in 44 BCE created a power vacuum, leading to a civil war. Octavian, later Augustus, emerged victorious, heralding the Roman Empire’s birth.
Under Augustus, the empire experienced the Pax Romana, a 200-year period of relative peace and stability. Augustus reformed the military, established efficient administrative systems, and initiated grand construction projects. The empire's borders expanded, encompassing territories from Britain to Egypt and from Spain to the Euphrates. Roman legions, renowned for their discipline and engineering prowess, secured and maintained these vast territories, building roads, fortifications, and cities that facilitated control and integration.
The Roman Empire’s society was hierarchical, with a rigid class system. At the top were the patricians, wealthy elites who held significant political power. Below them were the plebeians, free citizens with limited political influence, and the vast numbers of slaves who formed the backbone of the economy. The family unit was central, governed by the paterfamilias, the male head who held absolute authority.
Culturally, the Romans were eclectic, absorbing and adapting elements from the civilizations they encountered, particularly the Greeks. Roman art, literature, and philosophy reflected this synthesis, creating a rich cultural tapestry. Latin, the Roman language, became the lingua franca of the Western world, influencing numerous modern languages.
Roman architecture and engineering achievements were monumental. They perfected the arch, vault, and dome, constructing enduring structures like the Colosseum, Pantheon, and aqueducts. These engineering marvels not only showcased Roman ingenuity but also served practical purposes, from public entertainment to water supply.
How to Create Map Views in the Odoo 17 ERPCeline George
The map views are useful for providing a geographical representation of data. They allow users to visualize and analyze the data in a more intuitive manner.
2. Group Members
Arsalan Ali Daim (BSCS14068)
Danish Javed (BSCS14028)
Muhammad Hamza (BSCS14062)
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
3. Outline
1. What is Parallel Algorithm?
2. Its Abilities.
3. Why Parallel Computing?
4. Parallel Algorithms.
5. Limitations for Parallel Algorithms.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
4. What is Parallel Algorithm?
A parallel algorithm is an algorithm that has been
specifically written for execution on a computer with two or
more processors.
But it can be run on computers with single processor
(multiple functional units, pipelined functional units, pipelined memory systems)
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
5. What makes Parallel Algorithms better?
• Throughput: Is the number of operations done per time unit.
• Latency : Is the time needed to complete one operation.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
6. Why Parallel Computing?
The Real World is Massively Parallel:
• In the natural world, many complex, interrelated events are happening at the same time, yet within a
temporal sequence.
• Compared to serial computing, parallel computing is much better suited for modeling, simulating and
understanding complex, real world phenomena.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
7. Why Parallel Computing?
SOLVE LARGER / MORE COMPLEX PROBLEMS:
• Many problems are so large and/or complex that it is impractical or impossible to solve them on a
single computer, especially given limited computer memory.
• Example: Web search engines/databases processing millions of transactions per second
TAKE ADVANTAGE OF NON-LOCAL RESOURCES:
• Using computer resources on a wide area network, or even the Internet when local computer
resources are scarce or insufficient.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
8. Hardware implementation for
Parallel algorithms (PRAM MODEL)
In the PRAM model, processors communicate by reading from and writing to the shared
memory locations
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
9. Classification of PRAM MODEL
PRAM is classified in two basic types
1. CRAM(Concurrent RAM)
2. ERAM(Exclusive RAM)
And they also have some combinations.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
10. Parallel Algorithm
1. Odd – Even Transposition Sort
2. Parallel Merge Sort
3. Computing Sum of a Sequence with parallelism
There are many more…
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
11. Odd – Even Transposition Sort
Variation of bubble sort.
Operates in two alternating phases, even phase and odd phase.
Even phase
Even-numbered processes exchange numbers with their right
neighbors.
Odd phase
Odd-numbered processes exchange numbers with their
right neighbors.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
12. Odd – Even Transposition Sort – Example
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
Parallel time complexity: Tpar = O(n) (for P=n)
14. Odd – Even Transposition Sort
Assuming our array of n elements to sort is
very large, we will be working with many
virtual processors on the same processor
to emulate one Process per element.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
15. Merge Sort
Example of divide and conquer algorithms.
Sorting algorithm to sort a vector, first divide it into two parts.
Apply same method again to each subdivided part. When both are sorted with m and n
elements.
Merge them and it will produce sorted vector.
The average complexity T(n) = O(nlogn)
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
16. Parallel Merge Sort
Divided into two tasks:
1.Divide the list
2.Conquer the list
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
17. Parallel Merge Sort
Divide the list onto different processors
Simple tree structure like this:
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
18. Parallel Merge Sort
Merge elements as they come together.
Simple tree structure like this:
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
19. Parallel Merge Sort – Algorithm
ALGORITHM: mergesort(A)
1 if (|A| = 1) then return A
2 else
3 in parallel do
4 L := mergesort(A[0..|A|/2))
5 R := mergesort(A[|A|/2..|A|))
6 return merge(L, R)
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
20. Parallel Merge Sort – Complexity
Sequential Merge Sort = O(nlogn)
In Parallel, we have n processors
logn time required to divide sequence
logn time required to merge sequence
logn+logn = 2logn
So, T(n) = O(logn)
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
21. Computing the sum of a Sequence.
Consider a sequence of n elements.
Devise an algorithm that performs many operations in parallel.
In parallel, each element of A with an even index is paired and summed with the next element
of A. Like , A[0] is paired with A[1], A[2] with A[3], and so on.
The result is a new sequence of ⌈n/2⌉ numbers.
This pairing and summing step can be repeated until, after ⌈log2 n⌉ steps, a sequence consisting
of a single value is produced, and this value is equal to the final sum.
Sequentially, its time complexity is O(n) but using this technique of parallelism the time
complexity reduced to O(log2n).
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
22. The Limitations and Problems
• Data Dependency
• Race Condition
• Resource Requirements
• Scalability
• Parallel Slowdown
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
23. Data Dependency
Results from multiple use of the same location(s) in storage by different tasks.
e.g.
for (int i=0;i<100;i++)
array[i]=array[i-1]*20;
Shared memory architectures -synchronize read/write operations between tasks.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
24. Race Condition
If instruction 1B is executed between 1A and 3A, or if instruction 1A is executed between 1B and
3B, the program will produce incorrect data. This is known as a race condition.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
25. Resource Requirements
• The primary intent of parallel programming is to decrease execution wall clock time, however
in order to accomplish this, more CPU time is required. For example, a parallel code that runs
in 1 hour on 8 processors actually uses 8 hours of CPU time.
• The amount of memory required can be greater for parallel codes than serial codes, due to
the need to replicate data and for overheads associated with parallel support libraries and
subsystems.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
26. Scalability
Two types of scaling based on time to solution:
◦ Strong scaling: The total problem size stays fixed as more processors are added.
◦ Weak scaling: The problem size per processor stays fixed as more processors are added.
Hardware factors play a significant role in scalability. Examples:
◦ Memory-CPU bus bandwidth
◦ Amount of memory available on any given machine or set of machines
◦ Processor clock speed
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
27. Parallel Slowdown
• Not all parallelization results in speed-up.
• Once task split up into multiple threads those threads spend a large amount of time
communicating among each other resulting degradation in the system.
• This is known as parallel slowdown.
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)
28. Parallel Slowdown – Example
I have observed a few such attempts where parallel code used the Threading Building
Blocks library (TBB). Much to experimenters’ astonishment, not only their simple parallel
programs sometimes expose no reasonable speedup but even those can be slower than
sequential counterparts!
Conclusion: when developing programs with TBB, you should take into account that using TBB
classes and functions may impact compiler optimizations, which has especially bad impact on
simple algorithms with small amount of work per iteration. Proper use of local variables helps
optimization and improves parallel speedup.
For Further info: https://software.intel.com/en-us/blogs/2008/03/04/why-a-simple-test-can-
get-parallel-slowdown
PARALLEL ALGORITHM (DESIGN AND ANALYSIS OF ALGORITHMS)