Cost optimal algorithm definitions and Brent Theorem

1. AssociateProfessor,KGC-DehradunBy Dr. Heman Pathak

2. Definition 2.1:
Cost of an Algorithm
The cost of a PRAM computation is the
product of the parallel time complexity
and the number of processors used.
For example, a PRAM algorithm that has
time complexity (log p) using p
processors has cost (p log p).

3. Definition 2.2:
Cost Optimal Algorithm
A cost optimal parallel algorithm is
an algorithm for which the cost is in
the same complexity class as an
optimal sequential algorithm.

4. Definition: Cost Optimal
Problem
Sequential
Cost
Parallel Algorithm Cost
Optimal
No. of PEs Complexity Cost
Sum of n Numbers n n/2 log(n) nlog(n) No
Prefix Sum (n) n n-1 log(n) nlog(n) No
List Ranking n n log(n) nlog(n) No
Pre-order Tree
Traversal
n 2(n-1) log(n) nlog(n) No
Merging two sorted list n n log(n) nlog(n) No
Graph Coloring (n) Polynomial Cn n2 Cn X n2 No

5. Cost Optimal: Reduction Algorithm

6. Cost Optimal: Reduction Algorithm
 The total number of operations performed by the parallel
algorithm is of the same complexity class as an optimal sequential
algorithm.
 The parallel reduction algorithm performs about n/2 additions in
the first step, n/4 additions in the second step, n/8 additions in
the third step, and so on, executing a total of n -1 additions over
the log⁡( 𝑛) iterations.
 Since both the sequential and the parallel algorithms perform n-1
additions, a cost-optimal variant of the parallel reduction
algorithm may exists.

7. Cost Optimal: Reduction Algorithm
Cost = P * Time Complexity
= nlog(n)
Cost optimal = (n-1)
n-1 = P * log(n)
P = (n-1)/ log(n)
TC = log(n)
Cost = (n-1) same as Sequential
Once we have determined the appropriate number of processors, we
need to verify that there is indeed a cost-optimal parallel-reduction
algorithm with logarithmic time complexity.

8. Brent’s Theorem(2.2)(1974)

9. Brent’s Theorem Proof

11. Application to Parallel Reduction
Parallel Algorithm: Sum of n Numbers
• No. of Operations (M) = n-1
• Time Complexity (T) = log⁡( 𝑛)
• No. of PEs = 𝑛/2
Cost Optimal Parallel Algorithm
• No. of Operations (M) = n-1
• Time Complexity (T) = log⁡( 𝑛)
• No. of PEs (P) = (n-1)/ log(n) =⁡ 𝑛/log⁡( 𝑛)

12. Application to Parallel Reduction

13. Application to Parallel Reduction
n = 16
p = (n/log n) = 4

14. Application to Parallel Reduction
Steps No. of Op. General
1 9 n-1
2 8 n-2
3 6 n-4
4 2 n-8
: : :
i n-2i
Sequential Algorithm
for i = 1 to n-1
a[i] = a[i] + a[i-1]
endfor

15. Application to Parallel Reduction