The document discusses logical block protection, presenting previous results and benchmarks related to multithreading and CRC algorithms used in tape storage. It highlights performance metrics, illustrating that using only a few additional threads can maximize tape speed, with the CRC32-C algorithm showing potential for even greater speeds. The findings underscore the independence of CRC computations for each block, enabling increased efficiency through parallel computing.

1. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Tape Storage and CRC Protection
Summer Student Project 2014
Karel Ha
IT-DSS-TAB
CERN
1 / 17

2. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 / 17

3. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
2 / 17

4. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
3 Multithreading
2 / 17

5. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
3 Multithreading
4 Benchmarks
2 / 17

6. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Outline
1 Logical Block Protection
2 Previous results
3 Multithreading
4 Benchmarks
5 Conclusion
2 / 17

7. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Basic picture
3 / 17

8. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Used CRC algorithms
CRC algorithm supported on tapes:
4 / 17

9. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Used CRC algorithms
CRC algorithm supported on tapes:
Reed-Solomon
4 / 17

10. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Used CRC algorithms
CRC algorithm supported on tapes:
Reed-Solomon
CRC32-C
4 / 17

11. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
File verification of protection information
5 / 17

12. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Previous benchmarks
6 / 17

13. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Previous benchmarks
tape-write (no CRC protection)
cca. 250 MB/s
6 / 17

14. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Previous benchmarks
tape-write (no CRC protection)
cca. 250 MB/s
tape-write with Reed-Solomon CRC
cca. 130 MB/s
6 / 17

15. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Previous benchmarks
tape-write (no CRC protection)
cca. 250 MB/s
tape-write with Reed-Solomon CRC
cca. 130 MB/s
Reed-Solomon CRC with output to /dev/null
cca. 200 MB/s
6 / 17

16. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
CRC computations for every block are independent of one another.
7 / 17

17. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
CRC computations for every block are independent of one another.
Speed can be increased using parallel computing.
7 / 17

18. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
CRC computations for every block are independent of one another.
Speed can be increased using parallel computing.
This technique is applicable for any CRC algorithm!
7 / 17

19. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
CRC computations for every block are independent of one another.
Speed can be increased using parallel computing.
This technique is applicable for any CRC algorithm!
0
5
10
15
20
25
30
50 100 150 200 250 300 350 400 450 500
#threadsneeded
speed of the CRC algorithm (MB/s)
# threads needed to reach 250MB/s drive speed
7 / 17

20. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
CRC concurrently
Overview
CRCs
Compute
Threads
Input
Ouput
8 / 17

21. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Architecture
tpsrv212
Host: tpsrv212.cern.ch
Indexes: physical
Date: mer. 17 sept. 2014 11:30:15 CEST
Machine (16GB)
Socket P#0
L3 (8192KB)
L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB)
L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB)
L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB)
Core P#0 Core P#1 Core P#2 Core P#3
PU P#0
PU P#4
PU P#1
PU P#5
PU P#2
PU P#6
PU P#3
PU P#7
9 / 17

22. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Architecture
tpsrv100
Host: tpsrv100.cern.ch
Indexes: physical
Date: mer. 17 sept. 2014 11:31:05 CEST
Machine (63GB)
Socket P#0
L3 (15MB)
L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB) L2 (256KB)
L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB) L1d (32KB)
L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB) L1i (32KB)
Core P#0 Core P#1 Core P#2 Core P#3 Core P#4 Core P#5
PU P#0
PU P#6
PU P#1
PU P#7
PU P#2
PU P#8
PU P#3
PU P#9
PU P#4
PU P#10
PU P#5
PU P#11
10 / 17

23. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
100
200
300
400
500
600
700
800
900
0 2 4 6 8 10 12 14 16
MB/s
n threads
Reed-Solomon on tpsrv212, output to /dev/null
11 / 17

24. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
200
400
600
800
1000
1200
1400
0 5 10 15 20
MB/s
n threads
Reed-Solomon on tpsrv100, output to /dev/null
12 / 17

25. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
500
1000
1500
2000
2500
0 2 4 6 8 10 12 14 16
MB/s
n threads
CRC32-C on tpsrv212, output to /dev/null
13 / 17

26. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
500
1000
1500
2000
2500
3000
3500
0 5 10 15 20 25 30
MB/s
n threads
CRC32-C on tpsrv100, output to /dev/null
14 / 17

27. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
0
50
100
150
200
250
300
0 2 4 6 8 10 12 14 16
MB/s
n threads
Reed-Solomon on tpsrv212, output to tape
15 / 17

28. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Conclusion
Using only 1-2 additional threads, we are already able to reach
maximum tape speed.
16 / 17

29. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Conclusion
Using only 1-2 additional threads, we are already able to reach
maximum tape speed.
And CRC32-C algorithm promises even greater speeds!
16 / 17

30. Logical Block Protection Previous results Multithreading Benchmarks Conclusion
Thank you!
Merci!
17 / 17