The document analyzes the many-core scalability of file systems, highlighting challenges in utilizing multicore architectures efficiently for I/O operations. It evaluates existing file systems and identifies bottlenecks in scalability, revealing that many popular systems struggle under high contention and do not provide scalable performance. Conclusions emphasize the need for new designs to address these scalability issues, incorporating insights and methodologies for improvement.

1. Understanding Many-core Scalability
of File Systems
1
Changwoo Min et al.
Published in USENIX ATC’18
Van Giang Nguyen, Duc Canh Le

2. Contents
1
• Motivation
2
• Methodology
3
• Evaluation
4
• Conclusion
2

3. 3
Motivation

4. Motivation 1: Multicore machine’s era
▪ Multicore machine is popular today
▪ Multicore massively increases the computational capacity of
machine
▪ How about I/O system (ex. Storage)?
▪ In general, multicore can benefit from parallel I/O operations, but it
also produces many issues (e.g. data consistency)
4-16 cores 16-32 cores 32-64 cores
4

5. Motivation 2: Single CPU is not enough!
▪ Single CPU is not enough to fully utilize storage device at all
** IBM RamSan-70
* Intel, Performance Benchmarking for PCIe* and NVMe* Enterprise SSDs
5
▪ 00’s era, HDD (100-200 IOPS) → yes
▪ Now: SSD (so far 1M IOPS(**)) → no
▪ Disk is not always the bottleneck of system as before!!
Consider CPU run at 2.6GHz
→ ~10 us per I/O operation
→ ~100K I/O operations per sec (IOPS) / CPU
→ Is this enough?
On average(*):
27K CPU cycles per I/O

6. Motivation 3: Design or Implementation?
▪ How to tackle the problem
▪ New interfaces and system stacks, ex. NVMe can reduce to 10K cycles / IO operation
▪ Increase CPU frequency
▪ File system should be scalable
▪ Are existing file systems scalable?
▪ No in-depth study
▪ Need some insights to design scalable file systems
6

7. Research questions
7
▪ What file system operations are not scalable?
▪ Why they are not scalable?
▪ Is it the problem of implementation or design?

8. 8
Methodology

9. Technical challenges
9
▪ Applications are usually stuck with a few bottlenecks
→ cannot see the next level of bottlenecks before
resolving them
→ difficult to understand overall scalability behavior
▪ How to systematically stress file systems to understand scalability
behavior?

10. FXMARK
▪ Micro-benchmark:
▪ General file system operation
▪ Stress each component of FS
▪ Multi-process test
▪ Application benchmark
▪ Real-life applications
▪ I/O intensive
10

11. Work-flow
11

12. Experimental setup
Five most popular file
systems on Linux
Typical storage
platforms
12

13. 13
Evaluation

14. Expected Result
14

15. Are existing File Systems scalable?
▪ No, in most of benchmarks
15

16. In-depth Analysis
16

17. Evaluation – First scenario
17

18. Evaluation – First scenario
18

19. … only when not being accessed reference counter is working
19
Evaluation – First scenario

20. High contention on a page reference counter → Huge memory stall
20
Evaluation – First scenario

21. 21
Evaluation

22. 22
Evaluation – Second scenario

23. 23
Evaluation – Second scenario

24. 24
Evaluation – Second scenario

25. 25
Evaluation

26. 26
Evaluation – Third scenario

27. 27
Evaluation – Third scenario

28. 28
Evaluation

29. Findings
29
▪ High locality can cause performance collapse
▪ Overwriting could be as expensive as appending
▪ A file cannot be concurrently updated
▪ All directory operations are sequential
▪ Renaming is system-wide sequential
▪ Metadata changes are not scalable
▪ Non-scalability often means wasting CPU cycles
▪ Scalability is not portable See paper for details

30. 30
Evaluation – App Benchmark

31. 31
Evaluation – App Benchmark

32. Conclusion
32
▪ Comprehensive analysis of many-core scalability of five widely-
used file systems using FxMark
▪ Manycore scalability should be of utmost importance in file system
design
▪ New challenges in scalable file system design
▪ Minimizing contention, scalable consistency guarantee, spatial
locality, etc.