This document summarizes research on the latency performance of encoding with random linear network coding (RLNC). It finds that doubling the generation or symbol size increases latency by approximately 2x for a full vector encoding algorithm. Switching to an on-the-fly encoding algorithm provides more moderate latency gains of 1.41x for generation size 8, 1.62x for size 16, and 1.75x for size 32. Future work could confirm these results on different CPU architectures, further investigate results from 1kB to 1MB data sizes, and evaluate if multi-threading can decrease encoding latency.

1. Latency Performance of Encoding with
Random Linear Network Coding
Lars Nielsen 1,2,3
René Rydhof Hansen 1
Daniel E. Lucani 2,3
May 3, 2018
1Department Of Computer Science (Cassiopeia), Aalborg University
2Department of Engineering, Aarhus University
3Chocolate Cloud ApS

2. Problem
• User experience is
affected by long encoding
latency
• Large management
overhead, with multiple
generation
• What happens if we
change the NC
configuration?
• For what data size, is a
single generation
plausible?
1

3. Problem
• Can we confirm theory?
• Is it possible to predict changes to latency, based on NC
configuration?
Nistor et al. 2011, 2012 Focus: Latency of coefficient generation
from a Finite Field.
A. Paramanathan et al. 2013 and C. W. Sørensen et al. 2016
Focus: Energy consumption.
2

4. Network Coding
• A technique to improve the reliability, robustness, and security
of networks
• It is a subset of erasure correcting code
• Random Linear Network Coding (RLNC), is a subset of
Network Coding
• Companies and research have started to apply Network Coding
in data storage
3

5. Network Coding
4

6. Network Coding
4

7. Network Coding Encoding
• For each generation, a symbol matrix is constructed
• Each symbol represent a linear equation
• Coefficients are randomly drawn from a GF(2q)
C × S = CS
5

8. On-The-Fly
c1,1 c1,2 01,3 · · · 01,g ·








Symbol1
Symbol2
0
...
0g+r








= first coded symbol
6

9. On-The-Fly
c1,1 c1,2 01,3 · · · 01,g
c2,1 c2,2 c2,3 · · · 02,g
·











Symbol1
Symbol2
Symbol3
0
...
0g+r











= second coded symbol
6

10. On-The-Fly






c1,1 c1,2 01,3 · · · 01,g
c2,1 c2,2 c2,3 · · · 02,g
...
cg+r,1 cg+r,2 · · · cg+r,g






·








Symbol1
Symbol2
Symbol3
...
Symbolg








= g+r coded symbol
6

11. Experiment
Full Vector
f (g, k) = α · g · k(g + r) + β · k · (g − 1)(g + r)
On-The-Fly
h(g, k) = (α + β) · k · g−1
i=0 i + r · k · (α · g + β · (g − 1))
= (α+β)·k·g·(g−1)
2 + r · k · (α · g + β · (g − 1))
g = generation size, k = symbol size, r = amount of redundancy
symbols, α = cost of product operation and β = cost of addition
operation
7

12. Experiment Expectations
Algorithm / g 8 - 16 Result 16 - 32 Result
Full Vector 2.06x 2.03x
On-The-Fly 2.14x 2.06x
Data Sizes Expected
Latency
increase
Result
1kB - 1 MB 103x
1 - 10 MB 10x
10 - 20 MB 2x
2n MB ∼ 2x
g ∆L Result
8 2.14x
16 2.06x
32 2.03x
8

13. How the experiment was conducted
• The benchmark is implemented in C++ and uses Kodo
• Using multiple configuration for generation size and symbol
size
• Data is generated following Monte Carlo simulation
• We use high granularity clock for recording start/stop of
encoding
• For each experiment configuration 1000 experiments have
been performed
The source code, configurations, and results are available on
GitHub under the MIT software license
github.com/looopTools/sw9-source
9

14. For the Full Vector Algorithm
1kB 1MB 10MB 20MB 32MB 64MB 128MB 256MB 512MB
Data size
10−2
10−1
100
101
102
103
Latencyinms
10

15. For the Full Vector Algorithm
Data Sizes Expected Latency increase Result
1kB - 1 MB 103x ∼ 102x
1 - 10 MB 10x 13x
10 - 20 MB 2x 2x
2n MB ∼ 2x 1.92x − 2.41x, av-
erage 2.16x
1kB 1MB 10MB 20MB 32MB 64MB 128MB 256MB 512MB
Data size
10−2
10−1
100
101
102
103
Latencyinms
11

16. For the Full Vector Algorithm
1kB 1MB 10MB 20MB 32MB 64MB 128MB 256MB 512MB
Data size
10−2
10−1
100
101
102
103
Latencyinms
G=8
G=16
G=32
12

17. For the Full Vector Algorithm
Algorithm / g 8 - 16 Result 16 - 32 Result
Full Vector 2.06x 1.75x 2.03x 1.82x
1kB 1MB 10MB 20MB 32MB 64MB 128MB 256MB 512MB
Data size
10−2
10−1
100
101
102
103
Latencyinms G=8
G=16
G=32
13

18. Comparison of the On-The-Fly and Full Vector algorithms
1kB 1MB 10MB 20MB 32MB 64MB 128MB 256MB 512MB
Data size
10−2
10−1
100
101
102
103
Latencyinms
FV G=8
FV G=16
FV G=32
OTF G=8
OTF G=16
OTF G=32
14

19. Comparison of the On-The-Fly and Full Vector algorithms
g ∆L Result
8 2.14x 1.41x
16 2.06x 1.62x
32 2.03x 1.75x
1kB 1MB 10MB 20MB 32MB 64MB 128MB 256MB 512MB
Data size
10−2
10−1
100
101
102
103
Latencyinms
FV G=8
FV G=16
FV G=32
OTF G=8
OTF G=16
OTF G=32
15

20. Conclusion Future Work
Conclusion
• ∼ 2x increase to latency, when doubling generation- or symbol
size
• Gain of switching to On-The-Fly is more moderate than
expected
• 1.41x for g = 8
• 1.62x for g = 16
• 1.75x for g = 32
Future Work
• Confirm results on additional CPU architectures
• Further investigate the 1kB to 1MB results
• Can multi-threading be used to decrease latency?
16