The basics of the work-stealing tree data structure, presented at LCPC 2013.

1. Near Optimal Work-Stealing Tree for Highly Irregular Data-Parallel Workloads
Aleksandar Prokopec
Martin Odersky
1

2. Near Optimal Work-Stealing Tree for Highly Irregular Data-Parallel
Workloads
Aleksandar Prokopec
Martin Odersky
2

3. Uniform workload
(0 until 10000000) reduce (+)
3

4. Uniform workload
(0 until 10000000) reduce (+)
sum = sum + x
4

5. Uniform workload
(0 until 10000000) reduce (+)
sum = sum + x
…
N
cycles
5

6. Baseline workload
for (0 until 10000000) {}
…
N
cycles
6

7. Irregular workload
7

8. Irregular workload
N
cycles
8

9. Irregular workload
for {
x <- 0 until width
y <- 0 until height
} image(x, y) = compute(x, y)
N
cycles
9

10. Irregular workload
for {
x <- 0 until width
y <- 0 until height
} image(x, y) = compute(x, y)
N
cycles
10

11. Workload function
workload(n) – work spent on element n after the data-parallel operation completed
11

12. Workload function
Could be…
Runtime value
dependent
for {
x <- 0 until width
y <- 0 until height
} img(x, y) = compute(x, y)
workload(n) – work spent on element n after the data-parallel operation completed
12

13. Workload function
Could be…
Execution-schedule
dependent
for (n <- nodes)
n.neighbours += new Node
workload(n) – work spent on element n after the data-parallel operation completed
13

14. Workload function
Could be…
Totally random
for ((x, y) <- img.indices)
img(x, y) = sample(
x + random(),
y + random()
)
workload(n) – work spent on element n after the data-parallel operation completed
14

15. Data-parallel scheduler
Assign loop elements to workers
without knowledge about the workload function.
15

16. Data-parallel scheduler
1. Linear speedup for the baseline workload
Assign loop elements to workers
without knowledge about the workload function.
16

17. Data-parallel scheduler
1. Linear speedup for the baseline workload
2. Optimal speedup for irregular workloads
Assign loop elements to workers
without knowledge about the workload function.
17

18. Static batching
Decides on the worker-element assignment before the data-parallel operation begins.
N
cycles
18

19. Static batching
Decides on the worker-element assignment before the data-parallel operation begins.
No knowledge → divide uniformly.
Not optimal for even mildly irregular workloads.
N
cycles
19

20. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
progress
20

21. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
0
21

22. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
2
T0: CAS
T0
22

23. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
4
T1: CAS
T0
T1
23

24. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
6
T0: CAS
T0
T1
24

25. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
8
T0: CAS
T0
T1
25

26. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
10
T0: CAS
T0
T1
26

27. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
12
T0: CAS
T0
T1
27

28. Fixed-size batching
Workload-driven – decides during execution.
N
cycles
progress
Pros: lightweight
Cons: minimum batch size, contention
28

29. Fixed-size batching - contention
29

30. Factoring, GSS, TS
Batch size varies.
N
cycles
progress
Pros: lightweight
Cons: contention
30

31. Task-based work-stealing
N
cycles
0..2
2..4
4..8
8..16
31

32. Task-based work-stealing
N
cycles
0..2
2..4
4..8
8..16
2..4
4..8
8..16
T0
T1
0..2
32

33. Task-based work-stealing
N
cycles
0..2
2..4
4..8
8..16
2..4
4..8
8..16
T0
T1
0..2
steal – a rare event
33

34. Task-based work-stealing
N
cycles
0..2
2..4
4..8
8..16
2..4
4..8
8..16
T0
T1
10..12
12..16
8..10
0..2
34

35. Task-based work-stealing
Pros: can be adaptive - uses stealing information
Cons: heavyweight - minimum batch size much larger
N
cycles
0..2
2..4
4..8
8..16
2..4
4..8
8..16
T0
T1
10..12
12..16
0..2
8..10
35

36. Task-based work-stealing
N
cycles
0..2
2..4
4..8
8..16
Cannot be stolen
after T0 starts processing it
36

37. Work-stealing tree
0
0
T0
N
owned
37

38. Work-stealing tree
0
0
T0
N
0
50
T0
N
owned
owned
T0: CAS
38

39. Work-stealing tree
0
0
T0
N
0
50
T0
N
0
N
T0
N
…
owned
owned
completed
T0: CAS
T0: CAS
What about stealing?
39

40. Work-stealing tree
0
0
T0
N
0
50
T0
N
0
N
T0
N
…
owned
owned
completed
0
-51
T0
N
T0: CAS
T1: CAS
stolen
T0: CAS
40

41. Work-stealing tree
0
50
T0
N
0
N
T0
N
…
owned
completed
0
-51
T0
N
T0: CAS
stolen
T0: CAS
0
0
T0
N
owned
T1: CAS
41

42. Work-stealing tree
0
50
T0
N
0
N
T0
N
…
owned
completed
0
-51
T0
N
T0: CAS
stolen
0
-51
T0
N
expanded
50
50
T0
M
M
M
T1
N
T0: CAS
0
0
T0
N
owned
M = (50 + N) / 2
42

43. Work-stealing tree
0
50
T0
N
0
N
T0
N
…
owned
completed
0
-51
T0
N
T0: CAS
stolen
0
-51
T0
N
expanded
50
50
T0
M
M
M
T1
N
T0: CAS
0
0
T0
N
owned
M = (50 + N) / 2
T0 or T1: CAS
43

44. Work-stealing tree
0
50
T0
N
0
N
T0
N
…
owned
completed
0
-51
T0
N
T0: CAS
stolen
0
-51
T0
N
expanded
50
50
T0
M
M
M
T1
N
T0 or T1: CAS
T0: CAS
0
0
T0
N
owned
M = (50 + N) / 2
44

45. Work-stealing tree - contention
45

46. Work-stealing tree scheduling
1)find either a non-expanded, non-completed node
2)if not found, terminate
3)if not owned, steal and/or expand, and descend
4)advance until node is completed or stolen
5)go to 1)
50

47. Work-stealing tree scheduling
2)if not found, terminate
3)if not owned, steal and/or expand, and descend
4)advance until node is completed or stolen
5)go to 1)
1)find either a non-expanded, non-completed node
51

48. Choosing the node to steal
Find first, in-order traversal
2
9
5
3
52

49. Choosing the node to steal
Find first, in-order traversal
2
9
5
3
Catastrophic – a lot of stealing, huge trees
53

50. Choosing the node to steal
Find first, in-order traversal
Find first, random order traversal
2
9
5
3
2
9
5
3
Catastrophic – a lot of stealing, huge trees
54

51. Choosing the node to steal
Find first, in-order traversal
Find first, random order traversal
2
9
5
3
2
9
5
3
Catastrophic – a lot of stealing, huge trees
Works reasonably well.
55

52. Choosing the node to steal
Find first, in-order traversal
Find first, random order traversal
Find most elements
2
9
5
3
2
9
5
3
2
9
5
3
Catastrophic – a lot of stealing, huge trees
Works reasonably well.
Generates least nodes.
Seems to be best.
56

53. Comparison with fixed-size batching
57

54. Comparison with fixed-size batching
58

55. Comparison with task work-stealing
59

56. Thank you! Questions?
60

57. Finding work
61

58. Other workloads
62