The fundamental concept of Parallel Processing, Part -2. This is useful for CTEVT Diploma in Computer Engineering Students.

1. Distributed Computing
EG 3113 CT Diploma in Computer Engineering
5th Semester
Unit 1.2: Performance of Parallel processors
Lecture by : Er. Ashish K.C(Khatri)

2. Performance metrics of parallel processor:
• Speedup
• Efficiency
• Redundancy
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 2

3. Speedup:
• Speedup is a measure of performance.
• It measures the ratio between the sequential execution time and the parallel
execution time.
• The speedup is defined as the ratio of the serial runtime of the best sequential
algorithm for solving a problem to the time taken by the parallel algorithm to
solve the same problem on p processors.
• S(p)=
𝑇(1)
𝑇(𝑝)
where, T(1) – execution time with 1 processing unit
T(p) – execution time with p processing unit
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 3

4. • Figure: Example of speedup
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 4

5. Efficiency:
• Efficiency is a measure of the usage of the computational capacity.
• It measures the ratio between performance and the number of resources available
to achieve that performance.
• E(p)=
𝑆(𝑝)
𝑝
=
𝑇(1)
𝑝 ×𝑇(𝑝)
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 5

6. Redundancy:
• Redundancy measures the increase in the required computation when using more
processing units.
• It measures the ratio between the number of operations performed by the parallel
execution and by the sequential execution.
• R(p) =
𝑂(𝑝)
𝑂(1)
where, O(p) – total no. of operations performed by p processors
O(1) – total no. of operations performed by 1 processor
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 6

7. Amdahl’s Law:
• The speedup of a program using multiple processors in parallel computing is
limited by the time needed for the serial fraction of the problem.
• Suppose, Rajesh have to attend an invitation.
• Rajesh’s another two friend Radhe and Shyam are also invited.
• There are conditions that all three friends have to go there separately and all of
them have to be present at door to get into the hall.
• Now Rajesh is coming by car, Radhe by bus and Shyam is coming by foot.
• Now, how fast Rajesh and Radhe can reach there it doesn’t matter, they have to
wait for Shyam.
• So to speed up the overall process, we need to concentrate on the performance of
Shyam other than Rajesh or Radhe.
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 7

8. • Amdahl's law is often used in parallel computing to predict the theoretical speedup
when using multiple processors.
• For example, if a program needs 20 hours to complete using a single thread, but a
one-hour portion of the program cannot be parallelized,
• therefore only the remaining 19 hours (p = 0.95) of execution time can be
parallelized,
• then regardless of how many threads are devoted to a parallelized execution of this
program, the minimum execution time cannot be less than one hour.
• Hence, the theoretical speedup is limited to at most 20 times the single thread
performance
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 8

9. 8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 9

10. 8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 10

11. Numericals:
• Suppose one wants to determine if it is advantageous to develop a parallel version
of a certain application. Through experimentation, it was verified that 90% of the
execution time is spent in procedures that can be parallelizable. What is the
maximum speedup that can be achieved with a parallel version of the application
executing on 8 processing units?
• Solution: proportion that can be made parallel, P = 90% = 0.9
no. of processors, N = 8
then,
speedup , S(p) =
1
1−𝑃 +
𝑃
𝑁
=
1
1−0.9 +
0.9
8
=
1
0.1+0.1125
=
1
0.2125
≈ 4.71
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 11

12. • In an enhancement of a design of a CPU, the speed of a floating point unit has
been increased by 20% and the fixed point unit has been increased by 10%. What
is overall speedup achieved if the ratio of the no. of floating point operations to no.
of fixed point operation is 2:3 and floating point operation used to take twice the
time taken by the fixed point operation in original design.
Soln:
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 12

13. 8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 13
Numerical:

14. Numerical:
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 14

15. 8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 15

16. Gustafson’s law:
• Gustafson’s Law says that if you apply P processors to a task that has serial
fraction f, scaling the task to take the same amount of time as before, the
speedup is
• Speedup = P + f (1-P)
= f + P (1-f )
• While Amdahl’s law starts from the sequential execution time to estimate the
maximum speedup that can be achieved with multiple processing units,
• Gustafson-Barsis’ law does the opposite, i.e., it starts from the parallel execution
time to estimate the maximum speedup in comparison with the sequential
execution.
8/16/2022 Distributed Computing Notes © Er. Ashish K.C(Khatri) 16

17. Example:

19. End of Unit