The document provides an introduction to data-oriented design (DOD). It discusses how modern computer architectures have changed with memory latency now being the dominant factor over CPU speed. This has made the traditional object-oriented programming model focused on code inefficient. DOD is presented as an alternative paradigm that focuses on data transformations rather than objects. Key principles of DOD include keeping data close to CPU caches, processing data together in blocks, and separating data from code. Examples are given showing how DOD can improve performance and locality of reference. Benefits of DOD include maximum performance, ease of parallelization, and simpler code.

1. Introduction to
Data-Oriented Design
@YaroslavBunyak
Senior Software Engineer, SoftServe

2. Programming, M**********r
Do you speak it?

3. Story

4. Sieve of Eratosthenes
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

5. Sieve of Eratosthenes
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

6. Sieve of Eratosthenes
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

7. Sieve of Eratosthenes
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

8. Sieve of Eratosthenes
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

9. Sieve of Eratosthenes
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

10. Sieve of Eratosthenes
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

11. Sieve of Eratosthenes

12. Sieve of Eratosthenes
Simple algorithm

13. Sieve of Eratosthenes
Simple algorithm
Easy to implement

14. Sieve of Eratosthenes

15. Sieve of Eratosthenes
int array[SIZE];

16. Sieve of Eratosthenes
int array[SIZE];
array[i] = 1;

17. Sieve of Eratosthenes
int array[SIZE];
array[i] = 1;
if (array[i]) ...

18. Sieve of Eratosthenes
int array[SIZE];
array[i] = 1;
if (array[i]) ...

19. Sieve of Eratosthenes
int array[SIZE];
array[i] = 1;
if (array[i]) ...
int bits[SIZE / 32];

20. Sieve of Eratosthenes
int array[SIZE];
array[i] = 1;
if (array[i]) ...
int bits[SIZE / 32];
bits[i / 32] |= 1 << (i % 32);

21. Sieve of Eratosthenes
int array[SIZE];
array[i] = 1;
if (array[i]) ...
int bits[SIZE / 32];
bits[i / 32] |= 1 << (i % 32);
if (bits[i / 32] & (1 << (i % 32))) ...

22. Sieve of Eratosthenes

23. Sieve of Eratosthenes
Simple algorithm

24. Sieve of Eratosthenes
Simple algorithm
Easy to implement

25. Sieve of Eratosthenes
Simple algorithm
Easy to implement
But...

26. Sieve of Eratosthenes
Simple algorithm
Easy to implement
But...
unexpected results

27. Sieve of Eratosthenes

28. Sieve of Eratosthenes
The second implementation (bitset) is 3-5x
faster than first (array)

29. Sieve of Eratosthenes
The second implementation (bitset) is 3-5x
faster than first (array)
Even though it actually does more work

30. Why?!.

31. Fast Forward

32. ...

33. ...
• Years have passed

34. ...
• Years have passed
• I become a software engineer

35. ...
• Years have passed
• I become a software engineer
• And one day...

36. This Graph
CPU/Memory performance
Slide 17
Computer architecture: a quantitative approach
By John L. Hennessy, David A. Patterson, Andrea C. Arpaci-Dusseau

37. This Table
1980
Modern PC
Improvement, %
Clock speed, Mhz
6
3000
+500x
Memory size, MB
2
2000
+1000x
Memory bandwidth, MB/s
13
7000 (read)
2000 (write)
+540x
+150x
Memory latency, ns
225
~70
+3x
Memory latency, cycles
1.4
210
-150x

38. Our Programming
Model

39. Our Programming
Model
• High-level languages

40. Our Programming
Model
• High-level languages
• OOP

41. Our Programming
Model
• High-level languages
• OOP
• everywhere!

42. Our Programming
Model
• High-level languages
• OOP
• everywhere!
• objects scattered throughout the
address space

43. Our Programming
Model
• High-level languages
• OOP
• everywhere!
• objects scattered throughout the
address space
• access patterns are unpredictable

44. Meet
Data-Oriented Design

45. Ideas

46. Ideas
• Programs transform data

47. Ideas
• Programs transform data
• nothing more

48. Ideas
• Programs transform data
• nothing more
• Think about data, not code

49. Ideas
• Programs transform data
• nothing more
• Think about data, not code
• Hardware is not a black box

50. Program
data
xform
data

51. Program
data
xform
Your Program
data

52. Claim
• Memory latency is the king
• CPU cycles almost free

53. Memory

54. Memory
CPU
•
CPU registers

55. Memory
CPU
•
•
CPU registers
Cache Level 1
L1i
Cache
L1d
Cache

56. Memory
CPU
•
•
•
CPU registers
Cache Level 1
Cache Level 2
L1i
Cache
L1d
Cache
L2
Cache

57. Memory
CPU
•
•
•
•
CPU registers
Cache Level 1
Cache Level 2
L1i
Cache
L1d
Cache
L2
Cache
RAM
RAM

58. Memory
CPU
•
•
•
•
•
CPU registers
Cache Level 1
Cache Level 2
L1i
Cache
L1d
Cache
L2
Cache
RAM
RAM
HDD
Disk

59. Distance Metaphor

60. Distance Metaphor
•
L1 cache: it's on your desk, pick it up.

61. Distance Metaphor
•
•
L1 cache: it's on your desk, pick it up.
L2 cache: it's on the bookshelf in your office, get
up out of the chair.

62. Distance Metaphor
•
•
L1 cache: it's on your desk, pick it up.
•
Main memory: it's on the shelf in your garage
downstairs, might as well get a snack while you're
down there.
L2 cache: it's on the bookshelf in your office, get
up out of the chair.

63. Distance Metaphor
•
•
L1 cache: it's on your desk, pick it up.
•
Main memory: it's on the shelf in your garage
downstairs, might as well get a snack while you're
down there.
•
Disk: it's in, um, California. Walk there. Walk back.
Really.
L2 cache: it's on the bookshelf in your office, get
up out of the chair.

64. Distance Metaphor
•
•
L1 cache: it's on your desk, pick it up.
•
Main memory: it's on the shelf in your garage
downstairs, might as well get a snack while you're
down there.
•
Disk: it's in, um, California. Walk there. Walk back.
Really.
L2 cache: it's on the bookshelf in your office, get
up out of the chair.
http://hacksoflife.blogspot.com/2011/04/going-to-california-with-aching-in-my.html

65. Advice

66. Advice
• Keep your data closer to registers and
cache

67. Advice
• Keep your data closer to registers and
cache
• What’s good for memory - good for you

68. Example 1: AoS vs SoA
struct Tile
{
bool ready;
Data pixels; // big chunk of data
};
Tile tiles[SIZE];
vs
struct Image
{
bool ready[SIZE];
// hot data
Data pixels[SIZE]; // cold data
};

69. Example 1: AoS vs SoA
for (int i = 0; i < SIZE; ++i)
{
if (tiles[i].ready)
draw(tiles[i].pixels);
}
vs
for (int i = 0; i < SIZE; ++i)
{
if (image.ready[i])
draw(image.pixels[i]);
}

70. Example 1: AoS vs SoA
vs

71. Example 2: Existence
struct Image
{
bool ready[SIZE];
Data pixels[SIZE];
};
Image image;
vs
Data ready_pixels[N];
Data no_pixels[M];
// N + M = SIZE

72. Example 2: Existence
for (int i = 0; i < SIZE; ++i)
{
if (image.ready[i])
draw(image.pixels[i]);
}
vs
for (int i = 0; i < N; ++i)
{
draw(ready_pixels[i];
}

73. Example 3: Locality
std::vector<float> numbers;
float sum = 0.0f;
for (auto it : numbers)
sum += *it;
vs
std::list<float> numbers;
float sum = 0.0f;
for (auto it : numbers)
sum+ = *it;

74. Example 3: Locality
vs

75. Few Patterns
• A to B transform
• In place transform
• Existence based processing
• Data normalization
• DB design says hello!
• Task, gather, dispatch, and more...

76. Benefits of DOD

77. Benefits of DOD
•
Maximum performance
•
CPU doesn’t wait & starve

78. Benefits of DOD
•
•
Maximum performance
•
CPU doesn’t wait & starve
Easy to parallelize
•
•
data is grouped, transforms separated
ready for Parallel Processing, OOP doesn’t

79. Benefits of DOD
•
•
•
Maximum performance
•
CPU doesn’t wait & starve
Easy to parallelize
•
•
data is grouped, transforms separated
ready for Parallel Processing, OOP doesn’t
Simpler code
•
surprise!

80. References: Memory
• Ulrich Drepper “What Every Computer
Programmer Should Know About Memory”
• Крис Касперски “Техника оптимизации
програм. Еффективное использование
памяти”
• Christer Ericson “Memory Optimization”
• Igor Ostrovsky “Gallery of Processor Cache
Effects”

81. References: DOD
•
Noel Llopis “Data-Oriented Design”, Game Developer
Magazine, September 2009
•
Richard Fabian “Data-Oriented Desing”, book draft
http://www.dataorienteddesign.com/dodmain/
•
•
Tony Albrecht “Pitfalls of Object-Oriented Programming”
•
•
Mike Acton “Typical C++ Bullshit”
Niklas Frykholm “Practical Examples of Data Oriented
Design”
Data Oriented Design @ Google+

82. Thank You!

83. Q?