Philip Derbeko presents past design decisions that influenced the design of current filesystems, takes a look at how Linux tackles those problems and compares it with other operating systems, and discusses the upcoming revolution in storage and filesystem design.
Older than he looks, Philip Derbeko has been programming for over 20 years.
He was using Linux since days of Slackware 3.0 with kernel 2.0
Most of the years worked on storage, security systems and machine learning.
Currently, develops and herds a team of Linux, OS X and Windows kernel developers at enSilo.
16. How it is done
file
file
file
dentry
dentry
inode
inode
File attributes
Direct blocks
Direct blocks
…
Indirect
blocks
Double
Indirect
blocks
Triple
Indirect
blocks
DirectDirectDirectDirect
DirectDirectDirect
DirectDirectDirectDirect
17. Ext2 - fsck
•Unclean mount or mount counter
•Not everything can be solved
•Plan:
• Superblock check
• Free Blocks
• Inode sanity
• Inode links
• Duplicates
• Bad Blocks
• Directory checks
SLOW!!!
18. Other consistency options – Soft Updates
•Dependency Rules:
1. Never point to uninitialized structure
2. Never reuse before nullifying the pointers
3. Never reset an old pointer before a new one was set
24. Ext3 – Journal final comments
•Journal-assisted recovery: Redo Logging
•Commit Batching
•Journal Cleaning – mark the last checkpoint in journal superblock
•Deletes and reuse
25. Ext4
•Backward and forward compatible – up to a certain point
•Scalability
•“Sequentiality” improvements:
• Extent-based allocations
• Journal checksum speed up
• Delayed allocations
•Transparent Encryption
34. FS Patches (Linux 2.6 over 5079 patches)
•Maintenance (45%)
•Bugs (35%) – constant bug fixing over the life of FS
•Performance
•Reliability
•Features
FAST 2013 – “A Study of Linux File System Evolution”
35. Bug Consequences
•Corruption
•Crash
•Failure of operation
•Deadlock
•Hang
•Memory leak
•Other
FAST 2013 – “A Study of Linux File System Evolution”
38% of bugs are on
failure paths
frequency
36. “Timeline” – facts should not mess a story
Berkley
FFS
ext2
ext3
ReiserFS
ext4
ZFS
BTRFS
FAT
FAT32
NTFS
WinFS (dead)
ReFS
HFS
HFS+
APFS
37. What was not covered
•Shared, network, distributed and clustered file systems:
• WAFL
• AFS
• GFS and DFS
• WebDav
•Volume Management
•UnionFS (Knoppix CD+HDD, Docker layers)