The document discusses the UNIX file management system and how it uses a virtual file system (VFS) and virtual nodes (vnodes) to provide a common interface for accessing different file systems, with the VFS handling file system independent operations and vnodes representing files and delegating file system specific operations to each file system type's implementation. It also describes how UNIX uses buffer caches and disk schedulers to improve performance by reducing disk access through caching of recently or frequently used disk blocks in memory.
A presentation on the Ext4 file system and the evolution of Ext filesystem in Linux operating system. Linux uses virtual filesystem. The comparison of the ext filesystem generations is provided.
The document summarizes the evolution of Linux file systems from local to cluster to distributed systems. It discusses Ext2, Ext3, and Ext4 local file systems and improvements made to support larger file systems and reduce filesystem check times. It introduces cluster file systems used with shared storage for high availability and scaling compute and storage. Distributed file systems are described as scaling to unified storage across commodity hardware, with examples like HDFS based on the Google File System model with separate metadata and data servers. Current trends include further scaling out, flash technology use, and unified object/block/file storage.
The document describes the Linux Virtual File System (VFS) which provides a standard interface for file access across different file systems by handling file system calls and interacting with individual file systems. It discusses the key data structures in VFS including inodes, dentries, superblocks and the methods involved for operations like lookups, permission checks, attribute retrieval. The VFS acts as an abstraction layer that allows file operations to work consistently regardless of the underlying file system type.
The document summarizes the standard directory structure and purposes of the main directories in a Linux file system. The root directory (/) contains all other directories and files on the system. Key directories include /bin for essential executable binaries, /dev for device files, /etc for system configuration files, /home for user files, /lib for shared libraries, /sbin for system administration binaries, /tmp for temporary files, /usr for user programs and documentation, and /var for files that change frequently like logs.
Linux uses a unified, hierarchical file system to organize and store data on disk partitions. It places all partitions under the root directory by mounting them at specific points. The file system is case sensitive. The Linux kernel manages hardware resources and the file system, while users interact through commands interpreted by the shell. Journaling file systems like ext3 and ReiserFS were developed to improve robustness over ext2 by logging file system changes to reduce the need for integrity checks after crashes. Ext4 further improved on this with features like larger maximum file sizes and delayed allocation.
FUSE Developing Fillesystems in userspaceelliando dias
FUSE allows developing filesystems in userspace by acting as a bridge between the Linux kernel and filesystem implementations running in userspace. It works by passing filesystem operation requests like open, read, write from the kernel to a FUSE library in userspace. The library then calls functions implemented by the filesystem to handle each operation. A simple example filesystem is presented that implements basic functions like getattr in memory without directories. Further examples add functionality like initializing metadata, deleting files, and adding a hierarchical directory structure. Options are discussed for multithreading and concurrency support when processing concurrent operation requests.
The document discusses file systems, specifically the System V File System (S5FS) and UNIX File System (UFS). It provides details on how each represents files using inodes and disk maps, and how they allocate and manage disk blocks and free space. The goal of UFS was to lay out files to allow faster access and minimize wasted disk space compared to S5FS. Techniques used in UFS include larger block sizes, grouping related file data together, and allowing block fragmentation to reduce internal fragmentation.
The document discusses Python FUSE (Filesystem in Userspace) which allows users to create their own filesystems in userspace without modifying the kernel. It provides an overview of filesystem concepts, a brief history of filesystem development, and introduces FUSE as a way to develop filesystems using Python and other languages. Code examples are provided demonstrating how to create a basic hash table based filesystem in Python using the FUSE API.
A presentation on the Ext4 file system and the evolution of Ext filesystem in Linux operating system. Linux uses virtual filesystem. The comparison of the ext filesystem generations is provided.
The document summarizes the evolution of Linux file systems from local to cluster to distributed systems. It discusses Ext2, Ext3, and Ext4 local file systems and improvements made to support larger file systems and reduce filesystem check times. It introduces cluster file systems used with shared storage for high availability and scaling compute and storage. Distributed file systems are described as scaling to unified storage across commodity hardware, with examples like HDFS based on the Google File System model with separate metadata and data servers. Current trends include further scaling out, flash technology use, and unified object/block/file storage.
The document describes the Linux Virtual File System (VFS) which provides a standard interface for file access across different file systems by handling file system calls and interacting with individual file systems. It discusses the key data structures in VFS including inodes, dentries, superblocks and the methods involved for operations like lookups, permission checks, attribute retrieval. The VFS acts as an abstraction layer that allows file operations to work consistently regardless of the underlying file system type.
The document summarizes the standard directory structure and purposes of the main directories in a Linux file system. The root directory (/) contains all other directories and files on the system. Key directories include /bin for essential executable binaries, /dev for device files, /etc for system configuration files, /home for user files, /lib for shared libraries, /sbin for system administration binaries, /tmp for temporary files, /usr for user programs and documentation, and /var for files that change frequently like logs.
Linux uses a unified, hierarchical file system to organize and store data on disk partitions. It places all partitions under the root directory by mounting them at specific points. The file system is case sensitive. The Linux kernel manages hardware resources and the file system, while users interact through commands interpreted by the shell. Journaling file systems like ext3 and ReiserFS were developed to improve robustness over ext2 by logging file system changes to reduce the need for integrity checks after crashes. Ext4 further improved on this with features like larger maximum file sizes and delayed allocation.
FUSE Developing Fillesystems in userspaceelliando dias
FUSE allows developing filesystems in userspace by acting as a bridge between the Linux kernel and filesystem implementations running in userspace. It works by passing filesystem operation requests like open, read, write from the kernel to a FUSE library in userspace. The library then calls functions implemented by the filesystem to handle each operation. A simple example filesystem is presented that implements basic functions like getattr in memory without directories. Further examples add functionality like initializing metadata, deleting files, and adding a hierarchical directory structure. Options are discussed for multithreading and concurrency support when processing concurrent operation requests.
The document discusses file systems, specifically the System V File System (S5FS) and UNIX File System (UFS). It provides details on how each represents files using inodes and disk maps, and how they allocate and manage disk blocks and free space. The goal of UFS was to lay out files to allow faster access and minimize wasted disk space compared to S5FS. Techniques used in UFS include larger block sizes, grouping related file data together, and allowing block fragmentation to reduce internal fragmentation.
The document discusses Python FUSE (Filesystem in Userspace) which allows users to create their own filesystems in userspace without modifying the kernel. It provides an overview of filesystem concepts, a brief history of filesystem development, and introduces FUSE as a way to develop filesystems using Python and other languages. Code examples are provided demonstrating how to create a basic hash table based filesystem in Python using the FUSE API.
The document summarizes Linux file systems and input/output. It discusses:
1) Linux file systems arrange files on disk storage in a structured collection. Common file systems include Ext2, Ext3, Ext4, JFS, ReiserFS, XFS, and Btrfs.
2) Linux uses two caches for input/output - a page cache that is unified with virtual memory, and a buffer cache for metadata.
3) Devices are classified as block, character, or network. Block devices allow random access to fixed blocks, while character devices don't need to support regular file functionality. Network devices use the kernel's networking subsystem.
The document discusses the Python FUSE module, which allows users to create their own file systems in user space rather than kernel space. It provides an overview of what file systems are and some history. It then explains that FUSE allows non-privileged users to create their own file systems without editing kernel code by using a special file descriptor to communicate between user space and kernel space. Examples of simple file systems created with FUSE like a hash table based file system and ones that simulate large files or encrypt file contents are presented to demonstrate how FUSE can be used.
The document discusses Linux file systems. It provides an overview of Linux file system types including network file systems like NFS and SMB, and disk file systems like ext2, ext3, FAT32, and NTFS. It describes the physical structure of file systems on disk including the boot block, super block, inode list, and block list. It also summarizes the features and maximum sizes of different file system standards like ext2, ext3, ext4, ReiserFS, XFS, and JFS.
The document discusses Linux file systems and partitioning. It describes how to use the fdisk command to view and create partitions, and supported local file systems like Ext2, Ext3, Vfat, and ISO9660. It provides details on Ext3 file system structure, creation, conversion from Ext2, and tools like dumpe2fs, fsck, and tune2fs. It also covers mounting file systems using mount, automatic mounting from /etc/fstab, and unmounting file systems with umount.
Linux is a free, open-source operating system based on UNIX with a modular kernel. It uses processes, threads, virtual memory, and files systems. Device drivers allow access to hardware via the block I/O system. Interprocess communication includes signals, pipes, shared memory, and semaphores. Security features authentication via PAM and access controls permissions via user and group IDs.
The document discusses Linux file systems. It begins with an overview of file system architecture, including inodes, dentries, superblocks, and how data is never erased but overwritten. It then covers various local file systems like Ext2, Ext3, Ext4, ReiserFS, and XFS. Next it discusses log-structured and pseudo file systems. It also covers network file systems like NFS and CIFS. Finally it summarizes cluster, distributed, and Hadoop file systems. The document provides a technical overview of Linux file system types, structures, features and capabilities.
This document provides an overview of the Linux file system including:
1. It defines the main directories and contents according to the Filesystem Hierarchy Standard (FHS) with the root directory being "/" and possible multiple partitions and filesystems.
2. It describes the different types of files like ordinary files, directories, and special files as well as file permissions for reading, writing, and executing files and directories.
3. It explains how to change file permissions using the chmod command and navigate the file system using commands like pwd, cd, and ls including examples of using options, wildcards and navigation.
FUSE (Filesystem in Userspace) on OpenSolariselliando dias
FUSE (Filesystem in Userspace) allows implementing file systems in userspace programs. It consists of a FUSE userspace library and kernel module. The kernel module redirects VFS calls to the library via a character device. This allows non-privileged users to mount FUSE file systems after adding the proper privileges. Security concerns include privilege escalation, block device access for backed file systems, and access control enforcement.
The document summarizes files and directories from both the programmer and operating system perspectives. It discusses the file system abstraction, file naming conventions, file structures like streams of bytes and records, common file operations, file organization methods, access rights and permissions, and directory structures. It provides examples of different file systems like UNIX and highlights key concepts like relative and absolute pathnames.
Lesson 2 Understanding Linux File SystemSadia Bashir
The document provides an overview of Linux file systems and file types. It discusses:
1) The main types of files in Linux including directories, special files, links, sockets and pipes.
2) The standard Linux directory structure and the purpose of directories like /bin, /sbin, /etc, and /usr.
3) Common Linux file extensions and hidden files that begin with a dot.
4) Environment variables and how they can be used to customize a system.
5) Symbolic links and how they create references to files without copying the actual file.
FUSE and beyond: bridging filesystems slides by Emmanuel Dreyfuseurobsdcon
Abstract
Everything started with the desire to have a distributed filesystem better than NFS, which would be elastic and replicated with no SPOF. glusterFS was among the filesystems that could do that, but it uses The FUSE interface to the kernel, which was not available on NetBSD.
We will therefore discuss FUSE implementation and its NetBSD counterpart which is called PUFFS. A first but isufficient bridging attmpt called reFUSE was done, but porting real filesystems required the more complete PERFUSE bridge, which we will present here.
In a third part, we will look at the various unexpected roadblocks that had to be handled during PERFUSE implementation, in order to get a fully functionnal glusterFS on NetBSD.
Speaker bio
Emmanuel Dreyfus works as an IT manager at ESPCI ParisTech in France. He has been contributing for 13 years to NetBSD, in areas such as binary compatibility, and FUSE implementation. More at http://hcpnet.free.fr/pubz/
The document discusses Linux file systems. It describes that Linux uses a hierarchical tree structure with everything treated as a file. It explains the basic components of a file system including the boot block, super block, inode list, and block list. It then covers different types of file systems for Linux like ext2, ext3, ext4, FAT32, NTFS, and network file systems like NFS and SMB. It also discusses absolute vs relative paths and mounting and unmounting filesystems using the mount and umount commands.
This document provides an overview of the Linux filesystem, including its structure, key directories, and concepts like mounting. It describes the Filesystem Hierarchy Standard which defines the main directories and their contents. Key points covered include that everything in Linux is treated as a file, the top-level root directory is "/", essential directories like /bin, /dev, /etc, /home, /lib, /proc, /sbin, /usr, /var are explained, and mounting additional filesystems is described.
This document discusses Ubuntu file systems. It begins with an overview of Ubuntu's history and pros and cons. It then covers the basics of file systems, including what they are, common types (e.g. ext2, ext3), and how they are structured and mounted. It also discusses commands used to create, modify, and manage file systems.
The document describes the standard Linux filesystem hierarchy, including the purpose and some examples of the contents of the top-level directories like /bin, /boot, /dev, /etc, /home, /lib, /media, /mnt, /opt, /proc, /root, /sbin, /usr, and /var. Many directories contain essential system files and programs needed for booting, administration, and operation of the system, while others provide variable storage and mounting points for removable devices. The filesystem layout separates core operating system, user, and variable files for security and manageability.
The document discusses and compares different file systems, including FAT, FAT32, NTFS, and their key features and limitations. FAT is the oldest file system and was designed for small disks and simple structures. It uses a file allocation table to organize files. NTFS is proprietary to Windows and offers improvements like larger volume sizes, security features like encryption, compression and quotas. It also has better performance, especially on large volumes.
Goes into details about the prior knowledge required to design a file system. Concludes with a demo that explains how these concepts are used by Second Extended File System (ext2fs) A demo is also provided.
Presented at Florence, Italy on 6th December 2017
This document discusses disk and file system concepts including:
- Creating file systems using newfs and how it connects to mkfs
- Mounting file systems manually, via fstab, and using volume manager
- Identifying mounted file systems using mount, df, and mnttab
- Repairing file systems using fsck and handling recoverable vs unrecoverable damage
- Benefits of journaling file systems like reduced reboot time and data retention
A fast file system for unix presentation by parang saraf (cs5204 VT)Parang Saraf
This is the presentation of the paper "A fast file system for unix"
In case if you need a copy of these slides feel free to contact me at parang[DOT]saraf[AT]gmail
This document contains Linux commands for filesystem management, network management, system management, jobs and processes. It lists commands like df, du, fsck, mount, umount for managing filesystems and commands like arp, dig, finger, ftp, ifconfig, netstat, ping for network management. It also lists commands for managing system processes, users, time and shutdown.
The document summarizes the VFS/VNODE interface in the FreeBSD kernel, which provides a common interface for kernel subsystems to access filesystems without needing intimate knowledge of specific filesystem structures. It introduces the key concepts of the VFS and VNODE interfaces, which implement object-oriented polymorphism to allow filesystem-specific implementations while providing a standard interface. Common operations like mounting, opening files, and path lookups make use of virtual functions in the VFS and VNODE interfaces.
Mechanisms and tools of development and monitoring in Linux Kernel
A brief presentation about the tools and mechanisms about development for the Linux Kernel to help understanding of what is required.
Tools that create an execution profile or provide instrumentation through static or dynamic methods, the Linux Kernel code, will be presented.
Also discussed will be the GDB debugger and how through a remote virtual serial connection to a virtual machine it can be used to debug a live Kernel and Linux Kernel modules. Also demonstrated will be how a deeper understanding of the code can be attained by attaching the memory locations used by the Kernel module to the GDB session.
Lastly, some of the Kernel execution contexts, such as, interrupts, deferrable work, context, etc. are presented.
The document summarizes Linux file systems and input/output. It discusses:
1) Linux file systems arrange files on disk storage in a structured collection. Common file systems include Ext2, Ext3, Ext4, JFS, ReiserFS, XFS, and Btrfs.
2) Linux uses two caches for input/output - a page cache that is unified with virtual memory, and a buffer cache for metadata.
3) Devices are classified as block, character, or network. Block devices allow random access to fixed blocks, while character devices don't need to support regular file functionality. Network devices use the kernel's networking subsystem.
The document discusses the Python FUSE module, which allows users to create their own file systems in user space rather than kernel space. It provides an overview of what file systems are and some history. It then explains that FUSE allows non-privileged users to create their own file systems without editing kernel code by using a special file descriptor to communicate between user space and kernel space. Examples of simple file systems created with FUSE like a hash table based file system and ones that simulate large files or encrypt file contents are presented to demonstrate how FUSE can be used.
The document discusses Linux file systems. It provides an overview of Linux file system types including network file systems like NFS and SMB, and disk file systems like ext2, ext3, FAT32, and NTFS. It describes the physical structure of file systems on disk including the boot block, super block, inode list, and block list. It also summarizes the features and maximum sizes of different file system standards like ext2, ext3, ext4, ReiserFS, XFS, and JFS.
The document discusses Linux file systems and partitioning. It describes how to use the fdisk command to view and create partitions, and supported local file systems like Ext2, Ext3, Vfat, and ISO9660. It provides details on Ext3 file system structure, creation, conversion from Ext2, and tools like dumpe2fs, fsck, and tune2fs. It also covers mounting file systems using mount, automatic mounting from /etc/fstab, and unmounting file systems with umount.
Linux is a free, open-source operating system based on UNIX with a modular kernel. It uses processes, threads, virtual memory, and files systems. Device drivers allow access to hardware via the block I/O system. Interprocess communication includes signals, pipes, shared memory, and semaphores. Security features authentication via PAM and access controls permissions via user and group IDs.
The document discusses Linux file systems. It begins with an overview of file system architecture, including inodes, dentries, superblocks, and how data is never erased but overwritten. It then covers various local file systems like Ext2, Ext3, Ext4, ReiserFS, and XFS. Next it discusses log-structured and pseudo file systems. It also covers network file systems like NFS and CIFS. Finally it summarizes cluster, distributed, and Hadoop file systems. The document provides a technical overview of Linux file system types, structures, features and capabilities.
This document provides an overview of the Linux file system including:
1. It defines the main directories and contents according to the Filesystem Hierarchy Standard (FHS) with the root directory being "/" and possible multiple partitions and filesystems.
2. It describes the different types of files like ordinary files, directories, and special files as well as file permissions for reading, writing, and executing files and directories.
3. It explains how to change file permissions using the chmod command and navigate the file system using commands like pwd, cd, and ls including examples of using options, wildcards and navigation.
FUSE (Filesystem in Userspace) on OpenSolariselliando dias
FUSE (Filesystem in Userspace) allows implementing file systems in userspace programs. It consists of a FUSE userspace library and kernel module. The kernel module redirects VFS calls to the library via a character device. This allows non-privileged users to mount FUSE file systems after adding the proper privileges. Security concerns include privilege escalation, block device access for backed file systems, and access control enforcement.
The document summarizes files and directories from both the programmer and operating system perspectives. It discusses the file system abstraction, file naming conventions, file structures like streams of bytes and records, common file operations, file organization methods, access rights and permissions, and directory structures. It provides examples of different file systems like UNIX and highlights key concepts like relative and absolute pathnames.
Lesson 2 Understanding Linux File SystemSadia Bashir
The document provides an overview of Linux file systems and file types. It discusses:
1) The main types of files in Linux including directories, special files, links, sockets and pipes.
2) The standard Linux directory structure and the purpose of directories like /bin, /sbin, /etc, and /usr.
3) Common Linux file extensions and hidden files that begin with a dot.
4) Environment variables and how they can be used to customize a system.
5) Symbolic links and how they create references to files without copying the actual file.
FUSE and beyond: bridging filesystems slides by Emmanuel Dreyfuseurobsdcon
Abstract
Everything started with the desire to have a distributed filesystem better than NFS, which would be elastic and replicated with no SPOF. glusterFS was among the filesystems that could do that, but it uses The FUSE interface to the kernel, which was not available on NetBSD.
We will therefore discuss FUSE implementation and its NetBSD counterpart which is called PUFFS. A first but isufficient bridging attmpt called reFUSE was done, but porting real filesystems required the more complete PERFUSE bridge, which we will present here.
In a third part, we will look at the various unexpected roadblocks that had to be handled during PERFUSE implementation, in order to get a fully functionnal glusterFS on NetBSD.
Speaker bio
Emmanuel Dreyfus works as an IT manager at ESPCI ParisTech in France. He has been contributing for 13 years to NetBSD, in areas such as binary compatibility, and FUSE implementation. More at http://hcpnet.free.fr/pubz/
The document discusses Linux file systems. It describes that Linux uses a hierarchical tree structure with everything treated as a file. It explains the basic components of a file system including the boot block, super block, inode list, and block list. It then covers different types of file systems for Linux like ext2, ext3, ext4, FAT32, NTFS, and network file systems like NFS and SMB. It also discusses absolute vs relative paths and mounting and unmounting filesystems using the mount and umount commands.
This document provides an overview of the Linux filesystem, including its structure, key directories, and concepts like mounting. It describes the Filesystem Hierarchy Standard which defines the main directories and their contents. Key points covered include that everything in Linux is treated as a file, the top-level root directory is "/", essential directories like /bin, /dev, /etc, /home, /lib, /proc, /sbin, /usr, /var are explained, and mounting additional filesystems is described.
This document discusses Ubuntu file systems. It begins with an overview of Ubuntu's history and pros and cons. It then covers the basics of file systems, including what they are, common types (e.g. ext2, ext3), and how they are structured and mounted. It also discusses commands used to create, modify, and manage file systems.
The document describes the standard Linux filesystem hierarchy, including the purpose and some examples of the contents of the top-level directories like /bin, /boot, /dev, /etc, /home, /lib, /media, /mnt, /opt, /proc, /root, /sbin, /usr, and /var. Many directories contain essential system files and programs needed for booting, administration, and operation of the system, while others provide variable storage and mounting points for removable devices. The filesystem layout separates core operating system, user, and variable files for security and manageability.
The document discusses and compares different file systems, including FAT, FAT32, NTFS, and their key features and limitations. FAT is the oldest file system and was designed for small disks and simple structures. It uses a file allocation table to organize files. NTFS is proprietary to Windows and offers improvements like larger volume sizes, security features like encryption, compression and quotas. It also has better performance, especially on large volumes.
Goes into details about the prior knowledge required to design a file system. Concludes with a demo that explains how these concepts are used by Second Extended File System (ext2fs) A demo is also provided.
Presented at Florence, Italy on 6th December 2017
This document discusses disk and file system concepts including:
- Creating file systems using newfs and how it connects to mkfs
- Mounting file systems manually, via fstab, and using volume manager
- Identifying mounted file systems using mount, df, and mnttab
- Repairing file systems using fsck and handling recoverable vs unrecoverable damage
- Benefits of journaling file systems like reduced reboot time and data retention
A fast file system for unix presentation by parang saraf (cs5204 VT)Parang Saraf
This is the presentation of the paper "A fast file system for unix"
In case if you need a copy of these slides feel free to contact me at parang[DOT]saraf[AT]gmail
This document contains Linux commands for filesystem management, network management, system management, jobs and processes. It lists commands like df, du, fsck, mount, umount for managing filesystems and commands like arp, dig, finger, ftp, ifconfig, netstat, ping for network management. It also lists commands for managing system processes, users, time and shutdown.
The document summarizes the VFS/VNODE interface in the FreeBSD kernel, which provides a common interface for kernel subsystems to access filesystems without needing intimate knowledge of specific filesystem structures. It introduces the key concepts of the VFS and VNODE interfaces, which implement object-oriented polymorphism to allow filesystem-specific implementations while providing a standard interface. Common operations like mounting, opening files, and path lookups make use of virtual functions in the VFS and VNODE interfaces.
Mechanisms and tools of development and monitoring in Linux Kernel
A brief presentation about the tools and mechanisms about development for the Linux Kernel to help understanding of what is required.
Tools that create an execution profile or provide instrumentation through static or dynamic methods, the Linux Kernel code, will be presented.
Also discussed will be the GDB debugger and how through a remote virtual serial connection to a virtual machine it can be used to debug a live Kernel and Linux Kernel modules. Also demonstrated will be how a deeper understanding of the code can be attained by attaching the memory locations used by the Kernel module to the GDB session.
Lastly, some of the Kernel execution contexts, such as, interrupts, deferrable work, context, etc. are presented.
Kernel Recipes 2016 - New hwmon device registration API - Jean DelvareAnne Nicolas
The hwmon subsystem originates from the 1998 project lm-sensors. Along the way, there have been a lot of effort done to have all drivers present a standard interface to user-space, and consolidate the common plumbing into an easy-to-use, hard-to-get-wrong API. The final step of this long-running effort is happening right now.
Jean Delvare, SUSE
The document discusses file system implementation and structure. It covers topics like directory implementation using linear lists and hash tables, different allocation methods like contiguous, linked and indexed allocation, free space management using bit vectors and linked lists, improving performance through caching, and recovery methods like consistency checking and log structured file systems. It also provides an overview of the Sun Network File System (NFS) including its architecture, mount protocol, and remote operations.
This document provides an overview of file systems. It discusses the physical structure of a hard drive and how the master boot record partitions it into sections. File systems then organize data on those partitions by creating structures like directories, files and metadata. Common file systems include FAT, Ext, and others. File systems in Windows and Unix/Linux are compared. Journaling is introduced as a way to prevent data corruption.
Part 03 File System Implementation in LinuxTushar B Kute
Presentation on "Virtual File System Implementation in Linux".
Presented at Army Institute of Technology, Pune for FDP on "Basics of Linux Kernel Programming". by Tushar B Kute (http://tusharkute.com).
The document discusses disk scheduling algorithms. It describes seek time and rotational latency as the two main components of access time for disk drives. It then discusses the elevator algorithm, which schedules requests in one direction at a time to reduce seek times. This is similar to how an elevator works. The one-way elevator algorithm is also described, which only services requests in one direction to avoid starvation but requires an occasional long seek to return to the start.
The document provides information for students in BIOL 3397 about resources available at the university library, including over 2 million volumes held across three branch libraries. It outlines five class objectives related to understanding how to navigate the library website, locate peer-reviewed articles, and distinguish between primary, secondary, and tertiary literature. Services offered by the library that could aid research are also listed, such as remote access to journals and books, interlibrary loans, and printing allowances.
মাধ্যমিক স্তরে ভূগোল শিক্ষা ও করণীয়- মো: আবুল বাশার, সহকারী অধ্যাপক, ভূগোল, সরকারি টিচার্স ট্রেনিং কলেজ, সিলেট। দৈনিক ইত্তেফাক; ১৯ মে ২০১১- Md. Abul Bashar, Assistant Professor, Geography, Govt. Teachers' Training College, Sylhet.
Quickoffice Connect Mobile Suite is a productivity suite customized for the iPad that allows users to create, view, edit, and share Microsoft Office files. It includes a file manager to access local and cloud storage locations. Files can be opened, edited, and shared across various online services directly from the app. The suite also allows printing to AirPrint and PDF, as well as opening and editing email attachments.
The document provides details on several Federation starships from the Star Trek universe, including the USS Voyager, USS Prometheus, USS Enterprise-E, USS Enterprise-D, USS Enterprise-C, USS Defiant, USS Thunderchild, and USS Appalachia. Each entry includes the ship's class, registry number, maximum speed, armaments, defenses, propulsion systems, and basic dimensions. The ships range from exploration vessels like the Galaxy-class Enterprise-D to combat prototypes such as the Defiant-class USS Defiant.
Social Media and 2010 Olympic Winter GamesAshley Spilak
The document summarizes the Province of British Columbia's social media strategy for the 2010 Winter Olympics. The goals were to create engaging content to promote pride in BC and the Games, provide real-time information across multiple platforms, and build an online community for people to share their stories. The approach involved around-the-clock posting efforts during the Olympics. The strategy was successful in attracting over 100,000 website visits and thousands of social media followers who engaged with content like photos and videos from across the province. Lessons learned included sharing timely information, letting the community tell their own stories, being genuine, and using an editorial calendar.
জীবন দক্ষতাভিত্তিক শিক্ষার অন্তর্ভূক্ত একটি দক্ষতা -আত্মসচেতনতা অর্জনের সূত্র- Md. Abul Bashar, Assistant Professor, Geography, Govt. Teachers' Training College, Sylhet
1) The document describes Michael Kim's experiences during a trip to Qingyuan, China, where he participated in various outdoor activities including biking, rock climbing, catapult building, first aid training, camping, and caving.
2) Rock climbing was the hardest activity, but Michael was able to climb over 2 meters. Caving was the dirtiest experience where Michael almost lost his shoes in sticky mud.
3) Michael also visited a family's ancestral home, drew a soundscape of the environment, and taught English to curious local children, finding it a difficult but interesting task.
4) Overall, Michael enjoyed the trip and freedom from his parents, though he disliked the c
The Linux Kernel Implementation of Pipes and FIFOsDivye Kapoor
A walkthrough of the code structure used in the linux kernel to implement pipes and FIFOs.
This was presented to a Senior level class at the Indian Institute of Technology, Roorkee.
This document provides an introduction to file systems and the OCFS2 file system. It begins with basic concepts of how data can be stored using block devices, databases, and file systems. It then discusses file system interfaces, I/O models, and classifications. It provides an overview of the virtual file system (VFS) layer and its key data structures. It describes the EXT3 and OCFS2 file systems in detail, covering their layouts, journaling, mounting, and space management.
Introduction One of the key goals for the Windows Subsystem for Li.pdfanwarfoot
Introduction
One of the key goals for the Windows Subsystem for Linux is to allow users to work with their
files as they would on Linux, while giving full interoperability with files the user already has on
their Windows machine. Unlike a virtual machine, where you have to use network shares or
other solutions to share files between the host and guest OS, WSL has direct access to all your
Windows drives to allow for easy interop.
Windows file systems differ substantially from Linux file systems, and this post looks into how
WSL bridges those two worlds.
File systems on Linux
Linux abstracts file systems operations through the Virtual File System (VFS), which provides
both an interface for user mode programs to interact with the file system (through system calls
such as open, read, chmod, stat, etc.) and an interface that file systems have to implement. This
allows multiple file systems to coexist, providing the same operations and semantics, with VFS
giving a single namespace view of all these file systems to the user.
File systems are mounted on different directories in this namespace. For example, on a typical
Linux system your hard drive may be mounted at the root, /, with directories such as /dev, /proc,
/sys, and /mnt/cdrom all mounting different file systems which may be on different devices.
Examples of file systems used on Linux include ext4, rfs, FAT, and others.
VFS implements the various system calls for file system operations by using a number of data
structures such as inodes, directory entries and files, and related callbacks that file systems must
implement.
Inodes
The inode is the central data structure used in VFS. It represents a file system object such as a
regular file, directory, symbolic link, etc. An inode contains information about the file type, size,
permissions, last modified time, and other attributes. For many common Linux disk file systems
such as ext4, the on-disk data structures used to represent file metadata directly correspond to the
inode structure used by the Linux kernel.
While an inode represents a file, it does not represent a file name. A single file may have
multiple names, or hard links, but only one inode.
File systems provide a lookup callback to VFS which is used to retrieve an inode for a particular
file, based on the parent inode and the child name. File systems must implement a number of
other inode operations such as chmod, stat, open, etc.
Directory entries
VFS uses a directory entry cache to represent your file system namespace. Directory entries only
exist in memory, and contain a pointer to the inode for the file. For example, if you have a path
like /home/user/foo, there is a directory entry for home, user, and foo, each with a pointer to an
inode. Directory entries are cached for fast lookup, but if an entry is not yet in the cache, the
inode lookup operation is used to retrieve the inode from the file system so a new directory entry
can be created.
File objects
When an inode is opened, .
The document discusses several key topics about the FreeBSD operating system including:
- How to use the virtual consoles of FreeBSD and log into the system.
- An overview of UNIX file permissions and flags in FreeBSD.
- The default directory structure and disk organization of FreeBSD.
- How to mount and unmount file systems using the fstab file and mount/unmount commands.
- Concepts of processes, daemons, signals and killing processes.
- What shells are and how to change your default login shell.
This document provides an overview of Linux basics including:
- A brief history of Linux and how it originated from UNIX.
- An overview of Linux components including the kernel, userspace programs, shells, and how they interact.
- Instructions for installing Linux distributions like Red Hat, Debian, and SuSE.
- How to use basic Linux commands and work with files, directories, and processes.
- Where to find Linux documentation using commands like man and info.
It serves as an introductory guide to getting started with the Linux operating system.
The document provides an overview of distributed file systems, including NFS, AFS, Lustre, and others. It discusses key aspects like scalability, consistency, caching, replication, and fault tolerance. Lustre is highlighted as an example of a distributed file system that aims to remove bottlenecks and achieve high scalability through an object-based design with separate metadata and storage servers.
File systems organize and store data on various storage media like hard drives. They consist of structures like directories and files to track allocated space, file names and locations. Key functions include managing free space, directories, and file storage locations. Common file systems include FAT, NTFS, disk, flash, tape, database, network and special purpose file systems. File systems use inodes, directories, block allocation maps and other metadata to organize and track files.
XFS is a file system designed for large storage needs and high performance. It supports large files and directories through its use of extents to track file data locations. XFS provides features like dynamic inode allocation, extended attributes, disk quotas, and crash recovery through write-ahead logging to enable quick recovery of metadata after an unclean shutdown.
The document discusses file systems at three levels - hardware, kernel, and user space. It explains that a file system organizes data on storage devices (hardware level), provides drivers to access the data (kernel level), and presents a unified view to users through the root file system (/). It also summarizes key points about why file systems are needed, the relationships between partitions, file systems, and the root file system, and how to write a basic file system module that interacts with the virtual file system (VFS) in the kernel.
This document provides an overview of how XenServer manages storage supplied to virtual machines. It describes how Linux views physical disks and partitions, as well as logical volume management and network file systems. The key concepts covered include storage repositories, virtual disk images, storage attributes, and different storage types like local, NFS, Fibre Channel, and appliance-based storage. The target audience is IT specialists who need to understand XenServer's storage concepts and terminology for planning virtualized infrastructures.
Unix file systems 2 in unix internal systems senthilamul
The document discusses how UNIX organizes and accesses files on disk. It describes the file system structure, including inodes which contain metadata about each file, directories which map filenames to inodes, and block allocation which determines how file data is physically stored across disk blocks. It also covers subdirectories, hard and soft links, and comparisons of different file allocation strategies like contiguous, block, and extent-based allocation.
The Superuser: Root
Disks and Partitions
Making New Partitions
Mounting Filesystems
Mounting a Filesystem: mount
Mounting Other Filesystems
Unmounting a Filesystem: umount
Sample /etc/fstab
Filesystem Types
This document provides an overview of file system topics. It begins with an introduction to file systems and their relationship to operating system architecture. It then discusses the Virtual File System (VFS) interface and key metadata components like super blocks, inodes, and directory entries. The document reviews common file system optimizations based on memory hierarchy and storage characteristics. Examples of specific file systems are given, including Ext4, NTFS, ZFS, NFS, and Google File System. The document concludes by soliciting any questions.
Btrfs: Design, Implementation and the Current StatusLukáš Czerner
This document provides an overview of the Btrfs file system including its design, implementation, features and current status. Btrfs was created to address limitations in other Linux file systems like ext4 and XFS by providing improved scalability, reliability, and advanced features. It uses a copy-on-write design with checksumming and flexible btrees to store metadata and data. Key current features include snapshots, checksumming, scrubbing, and volume management. The file system is actively developed but aspects like the file system check and raid 5/6 support remain incomplete.
Disk and File System Management in LinuxHenry Osborne
This document discusses disk and file system management in Linux. It covers MBR and GPT partition schemes, logical volume management, common file systems like ext4 and XFS, mounting file systems, and file system maintenance tools. It also discusses disk quotas, file ownership, permissions, and the umask command for setting default permissions.
The document discusses Linux file systems. It describes that Linux uses a hierarchical tree structure with everything treated as a file. It explains the basic components of a file system including the boot block, super block, inode list, and block list. It then covers different types of file systems like ext2, ext3, ext4, FAT32, NTFS, and network file systems like NFS and SMB. It also discusses the physical structure of files on disk and basic Linux file system commands like mount, unmount, ls, and file paths.
The document discusses digital forensics and evidence extraction from NTFS computers. It describes how NTFS works, including features like the master file table, alternate data streams, volume shadow copies, and more. It explains that deleted or hidden data can potentially be uncovered by examining registry entries, volume shadow copies, unallocated space in the MFT, and clusters marked as bad.
This document discusses problems involving modulation techniques and signal detection. Problem 1 asks to sketch modulated waves for 8-ary ASK, 4-ary PSK, and 4-ary FSK modulation of a binary sequence. Problem 2 provides a formula for probability of error for 4-ary PAM and asks to calculate average power. Problem 3 does similarly for 4-ary QAM. Problem 4 describes a binary system with an integrate-and-dump detector and asks to analyze the detector and find minimum error probability.
This document contains 4 problems related to digital and analog communications techniques:
1) It describes Delta Modulation and how it works to encode analog signals, including issues with slope overload. It also introduces Adaptive Delta Modulation as an improvement.
2) It compares Differential Pulse Code Modulation to standard Pulse Code Modulation.
3) It provides exercises on encoding a binary sequence using BPSK, DPSK, and 4-ary ASK modulation. It also covers removing phase ambiguity in DPSK.
4) It presents a problem on time division multiplexing analog and digital sources at different data rates.
The document discusses problems related to analogue and digital communications. It contains 5 problems:
1) Drawing the spectrum of a message signal sampled at different rates and specifying the cutoff frequency to fully recover the original signal from its sampled version.
2) Sketching the resulting pulse code modulation (PCM) wave for one cycle of an input signal that is quantized using a 4-bit binary system.
3) Determining the number of quantizing levels, quantizer step size, average quantizing noise power, and output signal-to-noise ratio for a sinusoidal modulating wave quantized using an n-bit code word.
4) Finding the Nyquist sampling rate for two signals.
This document provides a tutorial on analogue and digital communications. It contains 5 questions about Fourier transforms and frequency domain analysis of signals. Question 1 asks about the Fourier transform and spectra of a continuous time signal. Question 2 finds the Fourier transform and spectra of another signal. Question 3 analyzes a signal composed of two sinusoidal components. Questions 4 and 5 determine the Fourier transforms of rectangular and truncated sinusoidal signals.
This document contains two questions about FM signals and phase-locked loops (PLL). Question 1 asks about the instantaneous frequency, modulation index, effective bandwidth using Carson's rule and universal rule, and Fourier transform of an FM signal. Question 2 provides a block diagram of a PLL and asks the student to show equations for the voltage controlled oscillator output, input to the VCO, and how the PLL locks into the input signal's phase. It also asks how the input to the VCO would be related to the message signal for an FM input.
1) The document discusses different modulation techniques including AM, DSB-SC, SSB, and VSB.
2) It provides examples of carrier and message signals for AM and DSB-SC modulation. Equations for the modulated signals and corresponding spectra are given.
3) Questions are asked about modulation index, bandwidth requirements, power efficiency for AM, and the block diagram and output for coherent detection of DSB-SC.
1) The document describes digital signal detection techniques at the receiver of a digital communication system.
2) It discusses the maximum a posteriori probability (MAP) and maximum likelihood (ML) detection criteria. The ML criterion reduces to choosing the signal that minimizes the Euclidean distance between the received signal vector and possible transmitted signals.
3) Detection errors occur when the received signal, distorted by noise, falls inside the decision region of another signal. The probability of error depends on the noise distribution around the actual transmitted signal.
1. This document discusses M-ary modulation techniques, which allow more than two amplitude, phase, or frequency levels to transmit more bits per symbol. This increases transmission rate or reduces bandwidth compared to binary modulation.
2. M-ary modulation techniques discussed include M-ASK, M-PSK, M-FSK, and M-QAM. M-ASK maps k bits to one of M amplitude levels. M-PSK maps k bits to one of M phase shifts of the carrier. M-QAM combines M-ASK with quadrature carriers to modulate both amplitude and phase.
3. Higher order modulation like M-QAM can significantly increase transmission rate but requires more transmission power and complex
The document provides an overview of digital passband modulation techniques. It discusses binary modulation schemes including amplitude-shift keying (ASK), frequency-shift keying (FSK), and phase-shift keying (BPSK). It also covers differential phase-shift keying (DPSK), which removes phase ambiguity in BPSK using differential encoding and decoding. Key aspects like signal representation, spectrum, and detection methods are described for each technique.
This document discusses an integrate-and-dump detector used in digital communications. It describes the operation of the integrate-and-dump detector, showing how it integrates the received signal plus noise over each symbol interval. The output of the integrator is used to detect whether a 1 or 0 was transmitted. An expression is derived for the probability of detection error in terms of the signal amplitude, noise power spectral density, and symbol interval. An example is also provided to calculate the error probability for a given binary signaling scheme and system parameters.
This document discusses quantization in analog-to-digital conversion. It describes how an analog signal is sampled, quantized by representing samples with discrete levels, and encoded into a digital signal. Quantization introduces noise that can be reduced by using more quantization levels or a smaller step size between levels. Non-uniform quantization allocates more levels to signal amplitudes that occur more frequently to improve efficiency compared to uniform quantization.
This document discusses multiplexing techniques used in communications systems. It describes frequency division multiplexing (FDM), time division multiplexing (TDM), and digital multiplexing used for digital telephone systems. FDM combines signals by assigning each a unique portion of the frequency spectrum. TDM combines signals by assigning each a unique time slot within a repeating time frame. Digital telephone systems use TDM to combine 24 digitized voice channels into a 193-bit frame transmitted every 125 microseconds.
This document discusses pulse code modulation (PCM) for analog to digital conversion. PCM involves sampling an analog signal, quantizing the sample values, and encoding the quantized levels into binary codes. The sampling rate must be at least twice the bandwidth of the analog signal. More bits per code provide higher quality but require more bandwidth. PCM is used in telephone systems with 8-bit coding at 8 kHz sampling, and in compact discs with 16-bit coding at 44.1 kHz sampling.
This document discusses delta modulation and adaptive delta modulation techniques for analogue to digital conversion.
Delta modulation encodes the difference between the input signal and a reference signal into a single bit per sample, creating a staircase-like approximation of the original signal. Adaptive delta modulation varies the step size according to the input signal level to prevent slope overload. Differential pulse code modulation encodes the difference between the current and predicted sample values, sending this difference value instead of absolute sample amplitudes.
This document discusses quantization in analog-to-digital conversion. It describes how an analog signal is sampled, quantized by representing samples with discrete levels, and encoded into a digital signal. Quantization introduces noise that can be reduced by using more quantization levels or a smaller step size between levels. Non-uniform quantization allocates more levels to signal amplitudes that occur more frequently to more efficiently represent the signal.
This document discusses quantization in analog-to-digital conversion. It describes how an analog signal is sampled, quantized by representing samples with discrete levels, and encoded into a digital signal. Quantization introduces noise that can be reduced by using more quantization levels or a smaller step size between levels. Non-uniform quantization allocates more levels to signal amplitudes that occur more frequently to more efficiently represent the signal.
This document discusses pulse modulation techniques in communications. It begins by reviewing continuous-wave modulation techniques studied previously, such as amplitude modulation and angle modulation. It then previews that pulse modulation will be studied next, including analog pulse modulation where a pulse feature varies continuously with the message, and digital pulse modulation using a sequence of coded pulses. The document provides explanations and equations regarding sampling of continuous-time signals, the sampling theorem, and recovery of the original analog signal from its samples. It also introduces pulse amplitude modulation (PAM) using natural and flat-top sampling, as well as pulse duration modulation (PDM) and pulse position modulation (PPM).
This document describes the operation of a phase-locked loop (PLL) circuit for demodulating frequency modulated (FM) signals. It contains 6 pages describing:
1) The basic block diagram of a PLL including a voltage controlled oscillator, multiplier, loop filter, and voltage controlled oscillator.
2) The mathematical equations showing how the PLL locks the phase and frequency of the voltage controlled oscillator to that of the incoming FM signal.
3) When the PLL is in phase lock and near phase lock based on the phase error between the signals.
4) How the PLL can be used to demodulate an FM signal and recover the original message signal by matching the phase and frequency of the voltage controlled
This document discusses the transmission of frequency modulated (FM) waves. It explains that the bandwidth of an FM wave is theoretically infinite but is effectively limited in practice. Carson's rule provides an estimate of the bandwidth but is not always accurate. The document also describes Armstrong's method for generating FM waves using narrowband phase modulation and frequency multiplication, and demodulating FM waves using envelope detection after taking the derivative of the received signal.
This document discusses angle modulation techniques for communications. It begins by defining phase modulation (PM) and frequency modulation (FM), where the carrier angle or frequency varies with the modulating signal. Key differences between PM and FM are outlined. Properties of angle modulation like constant transmitted power and irregular zero crossings are described. Narrowband FM is analyzed using sinusoidal modulation. Generation of narrowband FM using an integrator is also shown.
2. OS storage stack (recap)
Application
FD table
OF table
VFS
FS
Buffer cache
Disk scheduler
Device driver
2
3. Virtual File System (VFS)
Application
FD table
OF table
VFS
FS
Buffer cache
Disk scheduler
Device driver
3
4. Older Systems only had a single
file system
• They had file system specific open, close, read,
write, … calls.
• However, modern systems need to support
many file system types
– ISO9660 (CDROM), MSDOS (floppy), ext2fs, tmpfs
4
5. Supporting Multiple File
Systems
• Alternatives
– Change the file system code to understand
different file system types
• Prone to code bloat, complex, non-solution
– Provide a framework that separates file
system independent and file system
dependent code.
• Allows different file systems to be “plugged in”
5
6. Virtual File System (VFS)
Application
FD table
OF table
VFS
FS FS2
Buffer cache
Disk scheduler Disk scheduler
Device driver Device driver
6
7. Virtual file system (VFS)
/
open(“/home/leonidr/file”, …);
Traversing the directory hierarchy
ext3 may require VFS to issue requests
to several underlying file systems
/home/leonidr
nfs
7
8. Virtual File System (VFS)
• Provides single system call interface for many file
systems
– E.g., UFS, Ext2, XFS, DOS, ISO9660,…
• Transparent handling of network file systems
– E.g., NFS, AFS, CODA
• File-based interface to arbitrary device drivers (/dev)
• File-based interface to kernel data structures (/proc)
• Provides an indirection layer for system calls
– File operation table set up at file open time
– Points to actual handling code for particular type
– Further file operations redirected to those functions
8
9. The file system independent code
deals with vfs and vnodes
VFS FS
vnode inode
File system
dependent
File Descriptor
Tables
Open File Table code9 9
10. VFS Interface
• Reference
– S.R. Kleiman., "Vnodes: An Architecture for Multiple File System Types
in Sun Unix," USENIX Association: Summer Conference Proceedings,
Atlanta, 1986
– Linux and OS/161 differ slightly, but the principles are the same
• Two major data types
– vfs
• Represents all file system types
• Contains pointers to functions to manipulate each file system as a whole
(e.g. mount, unmount)
– Form a standard interface to the file system
– vnode
• Represents a file (inode) in the underlying filesystem
• Points to the real inode
• Contains pointers to functions to manipulate files/inodes (e.g. open, close,
read, write,…)
10
11. Vfs and Vnode Structures
struct vnode • size
Generic • uid, gid
(FS-independent) • ctime, atime, mtime
fields •…
fs_data
vnode ops
FS-specific
fields
ext2fs_read
ext2fs_write
...
• Block group number
FS-specific
• Data block list
implementation of
•…
vnode operations
11
12. Vfs and Vnode Structures
struct vfs
• Block size
Generic • Max file size
(FS-independent) •…
fields
fs_data
vfs ops
FS-specific
fields
ext2_unmount
ext2_getroot
...
• i-nodes per group
FS-specific
• Superblock address
implementation of
•…
FS operations
12
13. A look at OS/161’s VFS
Force the
The OS161’s file system type filesystem to
Represents interface to a mounted filesystem flush its content
to disk
struct fs { Retrieve the
int (*fs_sync)(struct fs *); volume name
const char *(*fs_getvolname)(struct fs *);
struct vnode *(*fs_getroot)(struct fs *); Retrieve the vnode
int (*fs_unmount)(struct fs *); associated with the
root of the
filesystem
void *fs_data;
}; Unmount the filesystem
Note: mount called via
function ptr passed to
Private file system
vfs_mount
specific data
13
14. Count the Number of
number of
“references”
Vnode times vnode
is currently
to this vnode open
struct vnode { Lock for mutual
exclusive
int vn_refcount; access to
int vn_opencount; counts
struct lock *vn_countlock;
struct fs *vn_fs; Pointer to FS
Pointer to FS
containing
void *vn_data; specific the vnode
vnode data
(e.g. inode)
const struct vnode_ops *vn_ops;
}; Array of pointers
to functions
operating on
vnodes 14
15. struct vnode_ops {
Vnode Ops
unsigned long vop_magic; /* should always be VOP_MAGIC */
int (*vop_open)(struct vnode *object, int flags_from_open);
int (*vop_close)(struct vnode *object);
int (*vop_reclaim)(struct vnode *vnode);
int (*vop_read)(struct vnode *file, struct uio *uio);
int (*vop_readlink)(struct vnode *link, struct uio *uio);
int (*vop_getdirentry)(struct vnode *dir, struct uio *uio);
int (*vop_write)(struct vnode *file, struct uio *uio);
int (*vop_ioctl)(struct vnode *object, int op, userptr_t data);
int (*vop_stat)(struct vnode *object, struct stat *statbuf);
int (*vop_gettype)(struct vnode *object, int *result);
int (*vop_tryseek)(struct vnode *object, off_t pos);
int (*vop_fsync)(struct vnode *object);
int (*vop_mmap)(struct vnode *file /* add stuff */);
int (*vop_truncate)(struct vnode *file, off_t len);
int (*vop_namefile)(struct vnode *file, struct uio *uio);
15
16. Vnode Ops
int (*vop_creat)(struct vnode *dir,
const char *name, int excl,
struct vnode **result);
int (*vop_symlink)(struct vnode *dir,
const char *contents, const char *name);
int (*vop_mkdir)(struct vnode *parentdir,
const char *name);
int (*vop_link)(struct vnode *dir,
const char *name, struct vnode *file);
int (*vop_remove)(struct vnode *dir,
const char *name);
int (*vop_rmdir)(struct vnode *dir,
const char *name);
int (*vop_rename)(struct vnode *vn1, const char *name1,
struct vnode *vn2, const char *name2);
int (*vop_lookup)(struct vnode *dir,
char *pathname, struct vnode **result);
int (*vop_lookparent)(struct vnode *dir,
char *pathname, struct vnode **result,
char *buf, size_t len);
};
16
17. Vnode Ops
• Note that most operations are on vnodes. How do we
operate on file names?
– Higher level API on names that uses the internal VOP_*
functions
int vfs_open(char *path, int openflags, struct vnode **ret);
void vfs_close(struct vnode *vn);
int vfs_readlink(char *path, struct uio *data);
int vfs_symlink(const char *contents, char *path);
int vfs_mkdir(char *path);
int vfs_link(char *oldpath, char *newpath);
int vfs_remove(char *path);
int vfs_rmdir(char *path);
int vfs_rename(char *oldpath, char *newpath);
int vfs_chdir(char *path);
int vfs_getcwd(struct uio *buf);
17
21. File Descriptors
• File descriptors
– Each open file has a file descriptor
– Read/Write/lseek/…. use them to specify
which file to operate on.
• State associated with a file fescriptor
– File pointer
• Determines where in the file the next read or write
is performed
– Mode
• Was the file opened read-only, etc….
21
22. An Option?
• Use vnode numbers as file descriptors
and add a file pointer to the vnode
• Problems
– What happens when we concurrently open
the same file twice?
• We should get two separate file descriptors and file
pointers….
22
23. An Option? Array of Inodes
fd in RAM
• Single global open
file array
– fd is an index into fp
the array i-ptr vnode
– Entries contain file
pointer and pointer
to a vnode
23
24. Issues
fd
• File descriptor 1 is
stdout
– Stdout is fp
• console for some v-ptr vnode
processes
• A file for others
• Entry 1 needs to be
different per
process!
24
25. Per-process File Descriptor
Array
• Each process has P1 fd
its own open file
array fp
– Contains fp, v-ptr v-ptr vnode
etc.
– Fd 1 can be any vnode
inode for each P2 fd
process (console,
log file). fp
v-ptr
25
26. Issue
• Fork
– Fork defines that the child P1 fd
shares the file pointer with
the parent
• Dup2 fp
– Also defines the file v-ptr vnode
descriptors share the file
pointer
• With per-process table, we vnode
can only have independent P2 fd
file pointers
– Even when accessing the
same file
fp
v-ptr
26
27. Per-Process fd table with global
open file table
• Per-process file descriptor
array P1 fd
– Contains pointers to open
file table entry
• Open file table array f-ptr fp
– Contain entries with a fp
and pointer to an vnode. v-ptr vnode
• Provides
– Shared file pointers if
required
fp
f-ptr vnode
– Independent file pointers P2 fd v-ptr
if required
• Example:
– All three fds refer to the f-ptr
same file, two share a file
pointer, one has an Per-process
independent file pointer File Descriptor 27
Open File Table
Tables
28. Per-Process fd table with global
open file table
• Used by Linux and
P1 fd
most other Unix
operating systems
f-ptr fp
v-ptr vnode
fp
f-ptr vnode
P2 fd v-ptr
f-ptr
Per-process
File Descriptor Open File Table 28
Tables
29. Buffer Cache
Application
FD table
OF table
VFS
FS
Buffer cache
Disk scheduler
Device driver
29
30. Buffer
• Buffer:
– Temporary storage used when transferring
data between two entities
• Especially when the entities work at different rates
• Or when the unit of transfer is incompatible
• Example: between application program and disk
30
31. Buffering Disk Blocks
• Allow applications to work with
arbitrarily sized region of a file
Application
Buffers – However, apps can still
Program in Kernel optimise for a particular block
size
RAM
Transfer of
arbitrarily
sized regions
Transfer of
whole
4 10
of file blocks 11
12 13 7
14
5 15
16 6
31
Disk
32. Buffering Disk Blocks
• Writes can return immediately
after copying to kernel buffer
Application
Buffers – Avoids waiting until write to
Program in Kernel disk is complete
– Write is scheduled in the
RAM background
Transfer of
arbitrarily
sized regions
Transfer of
whole
4 10
of file blocks 11
12 13 7
14
5 15
16 6
32
Disk
33. Buffering Disk Blocks
• Can implement read-ahead by
pre-loading next block on disk
Buffers into kernel buffer
Application
Program in Kernel – Avoids having to wait until
next read is issued
RAM
Transfer of
arbitrarily
sized regions
Transfer of
whole
4 10
of file blocks 11
12 13 7
14
5 15
16 6
33
Disk
34. Cache
• Cache:
– Fast storage used to temporarily hold data to
speed up repeated access to the data
• Example: Main memory can cache disk blocks
34
35. Caching Disk Blocks
• On access
Cached – Before loading block from disk,
blocks in check if it is in cache first
Application • Avoids disk accesses
Program Kernel
• Can optimise for repeated access
RAM for single or several processes
Transfer of
arbitrarily
sized regions
Transfer of
whole
4 10
of file blocks 11
12 13 7
14
5 15
16 6
35
Disk
36. Buffering and caching are
related
• Data is read into buffer; extra cache copy
would be wasteful
• After use, block should be put in a cache
• Future access may hit cached copy
• Cache utilises unused kernel memory
space; may have to shrink
36
37. Unix Buffer Cache
On read
– Hash the
device#, block#
– Check if match in
buffer cache
– Yes, simply use
in-memory copy
– No, follow the
collision chain
– If not found, we
load block from
disk into cache
37
38. Replacement
• What happens when the buffer cache is full and
we need to read another block into memory?
– We must choose an existing entry to replace
• Need a policy to choose a victim
– Can use First-in First-out
– Least Recently Used, or others.
• Timestamps required for LRU implementation
• However, is strict LRU what we want?
38
39. File System Consistency
• File data is expected to survive
• Strict LRU could keep critical data in
memory forever if it is frequently used.
39
40. File System Consistency
• Generally, cached disk blocks are prioritised in
terms of how critical they are to file system
consistency
– Directory blocks, inode blocks if lost can corrupt
entire filesystem
• E.g. imagine losing the root directory
• These blocks are usually scheduled for immediate write to
disk
– Data blocks if lost corrupt only the file that they are
associated with
• These blocks are only scheduled for write back to disk
periodically
• In UNIX, flushd (flush daemon) flushes all modified blocks to
disk every 30 seconds
40
41. File System Consistency
• Alternatively, use a write-through cache
– All modified blocks are written immediately to disk
– Generates much more disk traffic
• Temporary files written back
• Multiple updates not combined
– Used by DOS
• Gave okay consistency when
– Floppies were removed from drives
– Users were constantly resetting (or crashing) their machines
– Still used, e.g. USB storage devices
41
42. Disk scheduler
Application
FD table
OF table
VFS
FS
Buffer cache
Disk scheduler
Device driver
42
43. Disk Management
• Management and ordering of disk access
requests is important:
– Huge speed gap between memory and disk
– Disk throughput is extremely sensitive to
• Request order ⇒ Disk Scheduling
• Placement of data on the disk ⇒ file system
design
– Disk scheduler must be aware of disk
geometry
43
44. Disk Geometry
• Physical geometry of a disk with two zones
– Outer tracks can store more sectors than inner without exceed max information density
• A possible virtual geometry for this disk
44
45. Evolution of Disk Hardware
Disk parameters for the original IBM PC floppy disk and
a Western Digital WD 18300 hard disk
45
46. Things to Note
• Average seek time is approx 12 times better
• Rotation time is 24 times faster
• Transfer time is 1300 times faster
– Most of this gain is due to increase in density
• Represents a gradual engineering improvement
46
50. Disk Performance is Entirely Dominated
by Seek and Rotational Delays
• Will only get worse as
capacity increases much
faster than increase in Average Access Time Scaled to 100%
seek time and rotation 100%
speed
80%
– Note it has been easier
to spin the disk faster 60% Transfer
Rot. Del.
than improve seek time 40% Seek
• Operating System 20%
should minimise 0%
mechanical delays as 1
22
2
0.017
Transfer
much as possible Rot. Del. 100 4.165
Seek 77 6.9
Disk
50
51. Disk Arm Scheduling Algorithms
• Time required to read or write a disk
block determined by 3 factors
1. Seek time
2. Rotational delay
3. Actual transfer time
• Seek time dominates
• For a single disk, there will be a
number of I/O requests
– Processing them in random order leads
to worst possible performance
52. First-in, First-out (FIFO)
• Process requests as they come
• Fair (no starvation)
• Good for a few processes with clustered requests
• Deteriorates to random if there are many processes
53. Shortest Seek Time First
• Select request that minimises the seek time
• Generally performs much better than FIFO
• May lead to starvation
54. Elevator Algorithm (SCAN)
• Move head in one direction
– Services requests in track order until it reaches the last track,
then reverses direction
• Better than FIFO, usually worse than SSTF
• Avoids starvation
• Makes poor use of sequential reads (on down-scan)
• Less Locality
55. Modified Elevator (Circular SCAN, C-SCAN)
• Like elevator, but reads sectors in only one direction
– When reaching last track, go back to first track non-stop
• Better locality on sequential reads
• Better use of read ahead cache on controller
• Reduces max delay to read a particular sector