Data decomposition techniques

1. DECOMPOSITION
TECHNIQUES
By : Mohamed Ramadan

2. Agenda
1. DecompositionTechniques
1. Data Decomposition Introduction
1. PartitionTypes
2. Examples on PartitioningTypes
2. Exploratory Decomposition
1. The 15-puzzle problem
2. Parallel vs serial
3. Speculative Decomposition
4. Hybrid Decompositions
2. Characteristics ofTasks and Interactions

3. 1.Data Decomposition Introduction
Idea-partitioning of data leads to tasks
1. Powerful and commonly used method for deriving concurrency in algorithms that
operate on large data structures.
2. Decomposition of computations is done in two steps.
1. The data on which the computations are performed is partitioned
2. This data partitioning is used to induce a partitioning of the computations into
tasks.
3. The operations that these tasks perform on different data partitions are usually
similar

4. 1.1 Partition Types
The partitioning of data can be performed in many possible ways,
this critically impacts performance of a parallel algorithm.
1. Output Data Partitioning
2. Input Data Partitioning
3. Partitioning Input and Output Data
4. Intermediate Data Partitioning

5. 1.2.1 Output Data Partitioning
1. Often, each element of the output can be computed independently of others (but
simply as a function of the input).
2. A partition of the output across tasks decomposes the problem naturally.

6. 1.2.1 Output Data Partitioning
Consider the problem of multiplying two n x n matrices A and B to yield matrix C.The
output matrix C can be partitioned into four tasks as follows:

7. 1.2.1 Output Data Partitioning
A partitioning of output data does not result in a unique decomposition into tasks. For
example, for the same problem as in previous example, with identical output data
distribution, we can derive the following two (other) decompositions:

8. 1.2.1 Output Data Partitioning
Consider the problem of counting the instances of given itemset in a database of
transactions. In this case, the output (itemset frequencies) can be partitioned across
tasks.

10. 1.2.1 Output Data Partitioning
From the previous example, the following observations can be made:
• If the database of transactions is replicated across the processes, each task can be
independently accomplished with no communication.
• If the database is partitioned across processes as well (for reasons of memory
utilization), each task first computes partial counts.These counts are then aggregated
at the appropriate task.

11. 1.2.2 Input Data Partitioning
• Divide input into groups
• One task per group
• Get intermediate results
• Create one task to combine intermediate results
• Can partition input also partition input and output

12. Top-partition input Bottom-partition input and output

13. 1.2.3 Partitioning of Intermediate Data
• Computation can often be viewed as a sequence of transformation from the input to
the output data.
• Good for multi-stage algorithms
Multi-stage computations such that the output of one stage is the input to the subsequent stage.
• In previous example we have a maximum degree of concurrency of four
• We can increase the degree of concurrency by introducing an intermediate stage in
which eight tasks compute their respective product submatrices and store the results in a temporary
three-dimensional matrix D,The submatrix Dk,i,j is the product of Ai,k and Bk,j.
• Dk,i,j = Ai,k * Bk,j

16. Concurrency Picture
• Max concurrency of 8
• Max concurrency of 4 for output partition ( concurrency degree = 12 tasks )
• Price is storage for D

17. Concurrency Picture
• The Owner-Computes Rule A decomposition based on partitioning output or input data is also widely
referred to as the owner-computes rule.
• The idea behind this rule is that each partition performs all the computations involving data that it
owns. Depending on the nature of the data or the type of data-partitioning, the owner-computes rule
may mean different things. For instance, when we assign partitions of
• input data to tasks, then the owner-computes rule means that a task performs all the computations
that can be done using these data.
• On the other hand, if we partition the output data, then the owner-computes rule means that a task
computes all the data in the partition assigned to it.

18. 2. Exploratory Decomposition
• For search space type problems
• Partition search space into small parts
• Look for solution in each part
• Include a variety of discrete optimization problems

19. 2.1 Exploratory Decomposition
• Search Space ProblemThe 15 puzzle

20. • The state space can be explored by generating various successor states of the current state and to view
them as independent tasks.

21. 2.2 Parallel vs serial
It depends on where you find the answer
• The work performed by the parallel formulation can be either smaller or greater than that
performed by the serial algorithm. For example, consider a search space that has been
partitioned into four concurrent tasks as shown

22. 3. Speculative Decomposition
• Takes path before it knows result
• Win big or waste
• While one task is performing the computation whose output is used in deciding the next computation,
other tasks can concurrently start the computations of the next stage.This scenario is similar to
evaluating one or more of the branches of a switch statement in C in parallel before the input for the
switch is available.

23. 3. Speculative Decomposition
Exmple:
Parallel discrete event simulation
Idea: Compute results at c,d,e before output from a is known

24. 4. Hybrid
• Sometimes better to put two ideas together
• Quicksort - Recursion results in O(n) tasks, little concurrency.
• First decompose, then recurse

25. 2. Characteristics of Tasks and Interactions
• Once a problem has been decomposed into independent tasks, the characteristics of these tasks
critically impact choice and performance of parallel algorithms. Relevant task characteristics include:
• Task Generation
• Task Sizes
• Knowledge ofTask Sizes
• Size of Data Associated withTasks