SlideShare a Scribd company logo
1 of 27
© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
File System Modules
2© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
What to Expect?
W's of a File System
Three levels of File Systems
FS relation with a Hard Disk Partition
FS relation with /
Writing a FS module
3© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
What is a File System?
Three things at three levels
Hardware Space – The Physical Organization of Data
on the Storage Devices
Kernel Space – Drivers to decode & access the data
from the Physical Organization
User Space – All what you see from / - The User View
Which ever it be, it basically does
Organize our data
For easy access by tagging
For fast maintenance by optimizing
4© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Why we need a File System?
Preserve Data for Later
Access
Update
Discard
Tag Data for
Categorizing
Easy & Fast Lookup
Fast & Reliable Data Management by
Optimized Operations
5© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
The Three File Systems
Hardware (Space) File System - Storer
Kernel (Space) File System - Decoder
User (Space) File System - Show-er
Storer: Partition for the File System (/dev/*)
Creating the File System (mkfs.*)
Decoder: Driver for the File System (*.ko)
Show-er: Root File System (/xyz)
Connector of the 3: mount
Let's try an example
6© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Recall
7© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Understanding a Hard Disk
Example (Hard Disk)
Heads (or Platters): 0 – 9
Tracks (or Cylinders): 0 – 24
Sectors: 1 – 64
Size of the Hard Disk
10 x 25 x 64 x 512 bytes = 8000KiB
Device independent numbering
(h, t, s) → 64 * (10 * t + h) + s → (1 - 16000)
8© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Partitioning a Hard Disk
First Sector – Master Boot Record (MBR)
Contains Boot Info
Contains Physical Partition Table
Maximum Physical Partitions: 4
At max 1 as Extended Partition
Rest as Primary Partition
Extended could be further partitioned into
Logical Partitions
In each partition
First Sector – Boot Record (BR)
Remaining for File System / Format
Extended Partition BR contains the Logical Partition Table
9© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Placement of a Hardware FS
Each Partition contains a File System
Raw (no FS)
Organized
Organized one is initialized by the corresponding Formating
“Partition Type is to OS”
W95*, DOS*, BSD*, Solaris*, Linux, ...
“Format Type is to File System”
ext2, ext3, vfat, ntfs, jffs2, …
Particular Partition Types support only Particular File
System Types
10© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Design a FS
Let's Take 1 Partition
With 16 blocks of 2 sectors each
Size = 16 x 2 x 512 bytes = 16KiB
Given 3 data pieces
Source Code – 2.5KiB
Image – 8KiB
Document – 2KiB
Place optimally for further operations
11© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Kernel Space File System
Has 2 Roles to Play
Decode the Hardware FS Layout to access data (by
implementing your logic in s/w)
Interface that with the User Space FS to give you a unified
view, as you all see at /
First part: ext2, ext3, vfat, ntfs, jffs2, …
Also called the File System modules
We will use a simplified version
Second part: VFS
Kernel provides VFS to achieve the virtual user view
So, FS module should actually interact with VFS
12© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
VFS Specifics
Inherited from the Linux specific FS
Promoted to a super set of various FS
Providing a unified view to the User
Default values for various attributes
owner, group, permissions, …
File Types
Same seven types as in ext2/ext3
13© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Virtual File System Interactions
complete
vfat
VFS
ext2
Partition 0
....
Partition 1 ... Partition N ...
User Space File System View @ /
.... ....
14© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Virtual File System Internals
Super Block
Directory Entry Table
Inode Table
d
> File Name
> Inode Number
> File Meta Data
> File Data Block Nos
> FS Meta Data
> Root Inode Number
r
Data Block
15© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
VFS Interaction Internals
vfat
VFS
ext2
Buffer
Cache
Block
Drivers
Directory
Cache
Inode
Cache
....
Block
Drivers
16© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
VFS Translation Callbacks
Five categories
File System level
Super Block Operations
Super Block level (Meta Meta Data)
Inode Operations
Inode level (Meta Data)
Directory Entry Operations
File level (Data)
File Operations through the Buffer Cache
Buffer Cache level
Actual Data Operations
17© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
File System Registration
Header: <linux/fs.h>
APIs
int register_filesystem(file_system_type *)
int unregister_filesystem(file_system_type *)
File System Operations
get_sb/mount: (Invoked by mount)
get_sb_bdev/mount_bdev -> Fill the Super Block
kill_sb: (Invoked by umount)
kill_block_super
18© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Fill the Super Block
Block Size & Bits
Magic Number
Super Block Operations
File System type
Root Inode
Inode Operations
Inode Mode
File Operations
19© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Super Block Operations
Header: <linux/fs.h>
APIs
write_inode: Update File Meta Data
statfs: Get the File Status (Invoked by stat)
simple_statfs (Handler from libfs)
20© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Inode Operations
Header: <linux/fs.h>
APIs
lookup
Search the file name by absolute path
Traverse the inode chain
if found matching inode, then populate the dentry with the inode
information
Inode Information
Size
Mode
File Operations
...
21© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
File Operations
Header: <linux/fs.h>
APIs
open → generic_file_open
read → do_sync_read → generic_file_aio_read →
do_generic_file_read → readpage
write → do_sync_write → generic_file_aio_write →
generic_file_buffered_write → write_begin & write_end
release
readdir: Change Directory (cd). List Directory (ls)
Directory Entry
Fill Function
fsync: Sync the File (sync)
simple_sync_file
22© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Address Space Operations
The Buffer / Page Cache level Operations
Header: <linux/fs.h>
Page Operations
readpage (Invoked by read)
write_begin, write_end (Invoked by write)
writepage (Invoked by pdflush kernel threads)
Page Functions
PageUptodate, PageDirty, PageWriteback, PageLocked
SetPageUptodate, ClearPageDirty, unlock_page, ...
page_address, ...
23© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Dive into the Real Driver
© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
File System Pieces – Set 1
Hardware Space
File System Design (Logic) – real_sfs_ds.h
File System Creator (Application) – format_real_sfs.c
Kernel Space
File System Decoder (Driver) – real_sfs_ops.[hc]
File System Translator (Driver) – real_sfs.c
File System Show-er (VFS)
User Space
File System Connector (mount)
© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
File System Pieces – Set 2
Hardware Space
File System Design (Logic) – ddk_fs_ds.h
File System Creator (Application) – mkfs.ddkfs.c
Kernel Space
File System Decoder (Driver) – ddk_fs_ops.[hc]
File System Translator (Driver) – ddk_fs.c
File System Show-er (VFS)
User Space
File System Connector (mount)
26© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
What all have we learnt?
W's of a File System
Three levels of File Systems
And relation between them
FS relation with a Hard Disk Partition
FS relation with /
Role of VFS
Writing a FS module
Simplified Interaction with the Block Driver
5 Operation Sets with VFS
27© 2010-17 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Any Queries?

More Related Content

What's hot

Linux architecture
Linux architectureLinux architecture
Linux architecturemcganesh
 
Static partitioning virtualization on RISC-V
Static partitioning virtualization on RISC-VStatic partitioning virtualization on RISC-V
Static partitioning virtualization on RISC-VRISC-V International
 
Uboot startup sequence
Uboot startup sequenceUboot startup sequence
Uboot startup sequenceHoucheng Lin
 
linux software architecture
linux software architecture linux software architecture
linux software architecture Sneha Ramesh
 
Linux booting process!!
Linux booting process!!Linux booting process!!
Linux booting process!!sourav verma
 

What's hot (20)

PCI Drivers
PCI DriversPCI Drivers
PCI Drivers
 
Linux architecture
Linux architectureLinux architecture
Linux architecture
 
BeagleBone Black Bootloaders
BeagleBone Black BootloadersBeagleBone Black Bootloaders
BeagleBone Black Bootloaders
 
Static partitioning virtualization on RISC-V
Static partitioning virtualization on RISC-VStatic partitioning virtualization on RISC-V
Static partitioning virtualization on RISC-V
 
USB Drivers
USB DriversUSB Drivers
USB Drivers
 
Uboot startup sequence
Uboot startup sequenceUboot startup sequence
Uboot startup sequence
 
Embedded Android : System Development - Part II (Linux device drivers)
Embedded Android : System Development - Part II (Linux device drivers)Embedded Android : System Development - Part II (Linux device drivers)
Embedded Android : System Development - Part II (Linux device drivers)
 
Connect (4|n)
Connect (4|n)Connect (4|n)
Connect (4|n)
 
ARM and SoC Traning Part II - System
ARM and SoC Traning Part II - SystemARM and SoC Traning Part II - System
ARM and SoC Traning Part II - System
 
I2C Drivers
I2C DriversI2C Drivers
I2C Drivers
 
Aix overview
Aix overviewAix overview
Aix overview
 
Character Drivers
Character DriversCharacter Drivers
Character Drivers
 
linux software architecture
linux software architecture linux software architecture
linux software architecture
 
Embedded Android : System Development - Part IV
Embedded Android : System Development - Part IVEmbedded Android : System Development - Part IV
Embedded Android : System Development - Part IV
 
Explore Android Internals
Explore Android InternalsExplore Android Internals
Explore Android Internals
 
Linux booting process!!
Linux booting process!!Linux booting process!!
Linux booting process!!
 
DDoS対処の戦術と戦略
DDoS対処の戦術と戦略DDoS対処の戦術と戦略
DDoS対処の戦術と戦略
 
Xvisor: embedded and lightweight hypervisor
Xvisor: embedded and lightweight hypervisorXvisor: embedded and lightweight hypervisor
Xvisor: embedded and lightweight hypervisor
 
ARM and SoC Traning Part I -- Overview
ARM and SoC Traning Part I -- OverviewARM and SoC Traning Part I -- Overview
ARM and SoC Traning Part I -- Overview
 
Platform Drivers
Platform DriversPlatform Drivers
Platform Drivers
 

Viewers also liked (13)

Introduction to Linux Drivers
Introduction to Linux DriversIntroduction to Linux Drivers
Introduction to Linux Drivers
 
References
ReferencesReferences
References
 
Serial Drivers
Serial DriversSerial Drivers
Serial Drivers
 
Interrupts
InterruptsInterrupts
Interrupts
 
SPI Drivers
SPI DriversSPI Drivers
SPI Drivers
 
Embedded C
Embedded CEmbedded C
Embedded C
 
gcc and friends
gcc and friendsgcc and friends
gcc and friends
 
Kernel Programming
Kernel ProgrammingKernel Programming
Kernel Programming
 
Low-level Accesses
Low-level AccessesLow-level Accesses
Low-level Accesses
 
Audio Drivers
Audio DriversAudio Drivers
Audio Drivers
 
Video Drivers
Video DriversVideo Drivers
Video Drivers
 
BeagleBoard-xM Bootloaders
BeagleBoard-xM BootloadersBeagleBoard-xM Bootloaders
BeagleBoard-xM Bootloaders
 
File Systems
File SystemsFile Systems
File Systems
 

Similar to File System Modules

Similar to File System Modules (20)

File System Modules
File System ModulesFile System Modules
File System Modules
 
Introduction to Linux
Introduction to LinuxIntroduction to Linux
Introduction to Linux
 
Introduction to Linux
Introduction to LinuxIntroduction to Linux
Introduction to Linux
 
Shell Scripting
Shell ScriptingShell Scripting
Shell Scripting
 
File system
File systemFile system
File system
 
File Systems
File SystemsFile Systems
File Systems
 
Linux Basics
Linux BasicsLinux Basics
Linux Basics
 
Vfs
VfsVfs
Vfs
 
009709863.pdf
009709863.pdf009709863.pdf
009709863.pdf
 
Linux Memory Management
Linux Memory ManagementLinux Memory Management
Linux Memory Management
 
File
FileFile
File
 
Graphical user interface _ SEOSKILLS Hyderabad
Graphical user interface _ SEOSKILLS HyderabadGraphical user interface _ SEOSKILLS Hyderabad
Graphical user interface _ SEOSKILLS Hyderabad
 
Disk and File System Management in Linux
Disk and File System Management in LinuxDisk and File System Management in Linux
Disk and File System Management in Linux
 
Linux Internals Part - 1
Linux Internals Part - 1Linux Internals Part - 1
Linux Internals Part - 1
 
Linux file system
Linux file systemLinux file system
Linux file system
 
linuxfilesystem-180727181106 (1).pdf
linuxfilesystem-180727181106 (1).pdflinuxfilesystem-180727181106 (1).pdf
linuxfilesystem-180727181106 (1).pdf
 
Section02-Structures.ppt
Section02-Structures.pptSection02-Structures.ppt
Section02-Structures.ppt
 
Linux_Commands_MT.pdf
Linux_Commands_MT.pdfLinux_Commands_MT.pdf
Linux_Commands_MT.pdf
 
linux file sysytem& input and output
linux file sysytem& input and outputlinux file sysytem& input and output
linux file sysytem& input and output
 
File management
File managementFile management
File management
 

More from Anil Kumar Pugalia (20)

Kernel Debugging & Profiling
Kernel Debugging & ProfilingKernel Debugging & Profiling
Kernel Debugging & Profiling
 
Processes
ProcessesProcesses
Processes
 
System Calls
System CallsSystem Calls
System Calls
 
Embedded Software Design
Embedded Software DesignEmbedded Software Design
Embedded Software Design
 
Playing with R L C Circuits
Playing with R L C CircuitsPlaying with R L C Circuits
Playing with R L C Circuits
 
Mobile Hacking using Linux Drivers
Mobile Hacking using Linux DriversMobile Hacking using Linux Drivers
Mobile Hacking using Linux Drivers
 
Functional Programming with LISP
Functional Programming with LISPFunctional Programming with LISP
Functional Programming with LISP
 
Power of vi
Power of viPower of vi
Power of vi
 
"make" system
"make" system"make" system
"make" system
 
Hardware Design for Software Hackers
Hardware Design for Software HackersHardware Design for Software Hackers
Hardware Design for Software Hackers
 
RPM Building
RPM BuildingRPM Building
RPM Building
 
Linux User Space Debugging & Profiling
Linux User Space Debugging & ProfilingLinux User Space Debugging & Profiling
Linux User Space Debugging & Profiling
 
Linux Network Management
Linux Network ManagementLinux Network Management
Linux Network Management
 
System Calls
System CallsSystem Calls
System Calls
 
Timers
TimersTimers
Timers
 
Threads
ThreadsThreads
Threads
 
Synchronization
SynchronizationSynchronization
Synchronization
 
Processes
ProcessesProcesses
Processes
 
Signals
SignalsSignals
Signals
 
Linux File System
Linux File SystemLinux File System
Linux File System
 

Recently uploaded

Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 

Recently uploaded (20)

Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 

File System Modules

  • 1. © 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. File System Modules
  • 2. 2© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. What to Expect? W's of a File System Three levels of File Systems FS relation with a Hard Disk Partition FS relation with / Writing a FS module
  • 3. 3© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. What is a File System? Three things at three levels Hardware Space – The Physical Organization of Data on the Storage Devices Kernel Space – Drivers to decode & access the data from the Physical Organization User Space – All what you see from / - The User View Which ever it be, it basically does Organize our data For easy access by tagging For fast maintenance by optimizing
  • 4. 4© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Why we need a File System? Preserve Data for Later Access Update Discard Tag Data for Categorizing Easy & Fast Lookup Fast & Reliable Data Management by Optimized Operations
  • 5. 5© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. The Three File Systems Hardware (Space) File System - Storer Kernel (Space) File System - Decoder User (Space) File System - Show-er Storer: Partition for the File System (/dev/*) Creating the File System (mkfs.*) Decoder: Driver for the File System (*.ko) Show-er: Root File System (/xyz) Connector of the 3: mount Let's try an example
  • 6. 6© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Recall
  • 7. 7© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Understanding a Hard Disk Example (Hard Disk) Heads (or Platters): 0 – 9 Tracks (or Cylinders): 0 – 24 Sectors: 1 – 64 Size of the Hard Disk 10 x 25 x 64 x 512 bytes = 8000KiB Device independent numbering (h, t, s) → 64 * (10 * t + h) + s → (1 - 16000)
  • 8. 8© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Partitioning a Hard Disk First Sector – Master Boot Record (MBR) Contains Boot Info Contains Physical Partition Table Maximum Physical Partitions: 4 At max 1 as Extended Partition Rest as Primary Partition Extended could be further partitioned into Logical Partitions In each partition First Sector – Boot Record (BR) Remaining for File System / Format Extended Partition BR contains the Logical Partition Table
  • 9. 9© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Placement of a Hardware FS Each Partition contains a File System Raw (no FS) Organized Organized one is initialized by the corresponding Formating “Partition Type is to OS” W95*, DOS*, BSD*, Solaris*, Linux, ... “Format Type is to File System” ext2, ext3, vfat, ntfs, jffs2, … Particular Partition Types support only Particular File System Types
  • 10. 10© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Design a FS Let's Take 1 Partition With 16 blocks of 2 sectors each Size = 16 x 2 x 512 bytes = 16KiB Given 3 data pieces Source Code – 2.5KiB Image – 8KiB Document – 2KiB Place optimally for further operations
  • 11. 11© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Kernel Space File System Has 2 Roles to Play Decode the Hardware FS Layout to access data (by implementing your logic in s/w) Interface that with the User Space FS to give you a unified view, as you all see at / First part: ext2, ext3, vfat, ntfs, jffs2, … Also called the File System modules We will use a simplified version Second part: VFS Kernel provides VFS to achieve the virtual user view So, FS module should actually interact with VFS
  • 12. 12© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. VFS Specifics Inherited from the Linux specific FS Promoted to a super set of various FS Providing a unified view to the User Default values for various attributes owner, group, permissions, … File Types Same seven types as in ext2/ext3
  • 13. 13© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Virtual File System Interactions complete vfat VFS ext2 Partition 0 .... Partition 1 ... Partition N ... User Space File System View @ / .... ....
  • 14. 14© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Virtual File System Internals Super Block Directory Entry Table Inode Table d > File Name > Inode Number > File Meta Data > File Data Block Nos > FS Meta Data > Root Inode Number r Data Block
  • 15. 15© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. VFS Interaction Internals vfat VFS ext2 Buffer Cache Block Drivers Directory Cache Inode Cache .... Block Drivers
  • 16. 16© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. VFS Translation Callbacks Five categories File System level Super Block Operations Super Block level (Meta Meta Data) Inode Operations Inode level (Meta Data) Directory Entry Operations File level (Data) File Operations through the Buffer Cache Buffer Cache level Actual Data Operations
  • 17. 17© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. File System Registration Header: <linux/fs.h> APIs int register_filesystem(file_system_type *) int unregister_filesystem(file_system_type *) File System Operations get_sb/mount: (Invoked by mount) get_sb_bdev/mount_bdev -> Fill the Super Block kill_sb: (Invoked by umount) kill_block_super
  • 18. 18© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Fill the Super Block Block Size & Bits Magic Number Super Block Operations File System type Root Inode Inode Operations Inode Mode File Operations
  • 19. 19© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Super Block Operations Header: <linux/fs.h> APIs write_inode: Update File Meta Data statfs: Get the File Status (Invoked by stat) simple_statfs (Handler from libfs)
  • 20. 20© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Inode Operations Header: <linux/fs.h> APIs lookup Search the file name by absolute path Traverse the inode chain if found matching inode, then populate the dentry with the inode information Inode Information Size Mode File Operations ...
  • 21. 21© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. File Operations Header: <linux/fs.h> APIs open → generic_file_open read → do_sync_read → generic_file_aio_read → do_generic_file_read → readpage write → do_sync_write → generic_file_aio_write → generic_file_buffered_write → write_begin & write_end release readdir: Change Directory (cd). List Directory (ls) Directory Entry Fill Function fsync: Sync the File (sync) simple_sync_file
  • 22. 22© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Address Space Operations The Buffer / Page Cache level Operations Header: <linux/fs.h> Page Operations readpage (Invoked by read) write_begin, write_end (Invoked by write) writepage (Invoked by pdflush kernel threads) Page Functions PageUptodate, PageDirty, PageWriteback, PageLocked SetPageUptodate, ClearPageDirty, unlock_page, ... page_address, ...
  • 23. 23© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Dive into the Real Driver
  • 24. © 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. File System Pieces – Set 1 Hardware Space File System Design (Logic) – real_sfs_ds.h File System Creator (Application) – format_real_sfs.c Kernel Space File System Decoder (Driver) – real_sfs_ops.[hc] File System Translator (Driver) – real_sfs.c File System Show-er (VFS) User Space File System Connector (mount)
  • 25. © 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. File System Pieces – Set 2 Hardware Space File System Design (Logic) – ddk_fs_ds.h File System Creator (Application) – mkfs.ddkfs.c Kernel Space File System Decoder (Driver) – ddk_fs_ops.[hc] File System Translator (Driver) – ddk_fs.c File System Show-er (VFS) User Space File System Connector (mount)
  • 26. 26© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. What all have we learnt? W's of a File System Three levels of File Systems And relation between them FS relation with a Hard Disk Partition FS relation with / Role of VFS Writing a FS module Simplified Interaction with the Block Driver 5 Operation Sets with VFS
  • 27. 27© 2010-17 SysPlay Workshops <workshop@sysplay.in> All Rights Reserved. Any Queries?