Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

EE311 OS April 6, 2010


Published on

  • Be the first to comment

  • Be the first to like this

EE311 OS April 6, 2010

  1. 1. EE311 OS April 6, 2010 Prof. Kyu Ho Park Lecture 10:File Systems Implementation
  2. 2. File System Implementation <ul><li>New Trend- Storage Class Memory(SCM) </li></ul><ul><li>File-System Structure </li></ul><ul><li>File-System Implementation </li></ul><ul><li>Directory Implementation </li></ul><ul><li>Allocation Methods </li></ul><ul><li>Free-Space Management </li></ul>
  3. 3. New Memory Architecture <ul><li>Storage Class Memory(SCM) </li></ul>
  4. 4. System Evolution <ul><li>CPU-RAM-Disk </li></ul><ul><li>CPU-RAM-SSD-Disk </li></ul><ul><li>CPU-RAM-SCM-Disk </li></ul>
  5. 5. Hierarchy of Latency Freitas &Wilcke, IBM J.Res&Dev,2008 <ul><li>Disk 10E7-10E8 CPU cycles </li></ul><ul><li>SCM 10E3 </li></ul><ul><li>DRAM 10E2 </li></ul><ul><li>L2,L3 cache 10-100 </li></ul><ul><li>L1 1 </li></ul>
  6. 6. *SLC-Single level Cell, MLC-Multi Level Cell Roadmap for Memory Technology D.Roberts, T,Kgil, and T.Mudge, EDAA 2009 2009 2011 2013 2015 2017 NAND Flash-SLC* 0.0081 0.0052 0.0031 0.0021 0.0013 NAND Flash-MLC* 0.0041 0.0013 0.0008 0.0005 0.0003 PCRAM(nMOSFET)-SLC* 0.0254 0.0123 0.0069 0.0036 0.0024 PCRAM(nMOSFET)-MLC* 0.0127 0.0061 0.0017 0.0009 0.0006 DRAM Cell density 0.0153 0.0096 0.0061 0.0038 0.0024 Flash write/erase cycles 1E+05 1E+05 1E+05 1E+05 1E+05 PCRAM write/erase cycles 1E+10 1E+10 1E+12 1E+15 1E+15 Flash SLC/MLC data retention (years) 10-20 10-20 10-20 10-20 10-20 PCRAM SLC/MLC data retention (years) >10 >10 >10 >10 >10
  7. 7. New Memory Architectures NVRAMOS 2009 <ul><li>An Empirical Study using NVRAM </li></ul><ul><ul><li>Performance/Energy tradeoffs on NVRAM </li></ul></ul><ul><ul><li>Operating System Support for NVRAM </li></ul></ul><ul><ul><li>Green data center with NVRAM </li></ul></ul>RAM-Flash RAM-SCM SCM - Flash SCM - Only < Best power efficiency > < Best Performance > CPU SCM CPU SCM NOR, NAND Flash CPU RAM SCM CPU RAM NOR, NAND Flash
  8. 8. New Memory Architectures NVRAMOS 2009 <ul><li>I/O bound job </li></ul><ul><ul><li>SCM is best </li></ul></ul><ul><li>Memory bound job </li></ul><ul><ul><li>DRAM is best </li></ul></ul><ul><li>CPU bound job </li></ul><ul><ul><li>Little impacts </li></ul></ul>
  9. 9. SCM only System NVRAMOS 2009 <ul><li>SCM </li></ul><ul><ul><li>Great potential to reduce energy consumption </li></ul></ul><ul><ul><li>SCM as memory cause performance degradation </li></ul></ul>SCM - Only CPU SCM
  10. 10. Operating System support for SCM NVRAMOS 2009 <ul><li>Operating System support for SCM </li></ul><ul><ul><li>Unified file object and memory objects </li></ul></ul><ul><ul><li>Eliminate redundant I/O accesses </li></ul></ul><ul><ul><li>mmap like operations for all memory/disk data </li></ul></ul>Disk - Twice accesses for same data to update Buffer cache File SCM File memory
  11. 11. Adaptive Context Switch NVRAMOS 2009 <ul><li>Context Switch on Block devices </li></ul><ul><ul><li>Fast block devices with SCM </li></ul></ul><ul><ul><ul><li>No need to context switch, which takes 100us </li></ul></ul></ul><ul><ul><ul><li>Keep use whole schedule quantum </li></ul></ul></ul><ul><ul><li>5~10% performance improvements </li></ul></ul><ul><ul><li>Shared object accesses in multiple threads cause performance degradation/malfunctions(Why?) </li></ul></ul>I/O access less than 1ms CPU RAM SCM
  12. 12. Objectives <ul><li>To describe the details of implementing local file systems and directory structures </li></ul><ul><li>supplementary to Lecture 9. </li></ul>
  13. 13. File-System Structure <ul><li>File structure </li></ul><ul><ul><li>Logical storage unit </li></ul></ul><ul><ul><li>Collection of related information </li></ul></ul><ul><li>File system resides on secondary storage (disks) </li></ul><ul><li>File system organized into layers </li></ul><ul><li>File control block – storage structure consisting of information about a file </li></ul>
  14. 14. File Systems <ul><li>How the file system can be looked to the user; </li></ul><ul><ul><li>Define a file and its attributes , the operations allowed on a file, and the directory structure for organizing a file. </li></ul></ul><ul><li>Algorithms and data structures to map the logical file system to the physical disk. </li></ul>
  15. 15. Layered F ile System
  16. 16. Layered File System <ul><li>I/O control; </li></ul><ul><li>It consists of device drivers and interrupt handlers to transfer information between the main memory and the disk system. </li></ul><ul><li>Device drivers: Its input consists of high-level commands. </li></ul><ul><li>“ get block 1000 ” “ H/W specific instructions” </li></ul>Device Driver
  17. 17. Basic File System Layer <ul><li>It issues generic commands to the appropriate device drivers to read and write physical blocks on the disk. </li></ul><ul><li>Each physical block is identified by its numeric disk address( drive 1, cylinder 3, track 2, sector 10) </li></ul>
  18. 18. File organization module <ul><li>It knows about files and their logical blocks as well as physical blocks. </li></ul><ul><li>It can translate logical block addresses to physical block addresses for the basic file system. </li></ul>
  19. 19. Logical File System <ul><li>It manages metadata information. </li></ul><ul><ul><li>Metadata : It includes all of the file system structure except the actual data. </li></ul></ul><ul><li>It manages the directory structure to provide the file organization information with the information. </li></ul><ul><li>How? </li></ul>
  20. 20. Why layered structure?
  21. 21. File System Overview[ On Disk] <ul><li>Boot block: It contains information needed by the system to boot an operating system. </li></ul><ul><li>Volume control block: </li></ul><ul><li>It contains volume( or partition) details, such as the number of blocks in the partition, size of blocks, free-block count and free-block pointers, free FCB count and FCB pointers. </li></ul><ul><li>In UNIX, it is called a superblock. </li></ul>
  22. 22. File System Overview <ul><li>A directory structure per file system: </li></ul><ul><li>In UNIX, it includes file names and associated inode numbers. </li></ul><ul><li>A per-file FCB: </li></ul><ul><li>In UNIX, it is called i-node </li></ul>
  23. 23. In-memory <ul><li>An in-memory mount table: </li></ul><ul><ul><li>It contains information about each mounted volume. </li></ul></ul><ul><li>An in-memory directory-structure cache: </li></ul><ul><li>It holds the directory information of recently accessed directories. </li></ul><ul><li>System-wide open-file table: </li></ul><ul><li>It contains a copy of the FCB of each open file. </li></ul><ul><li>Per-process open-file table: </li></ul><ul><li>It contains a pointer to the appropriate entry in the system-wide open-file table. </li></ul>
  24. 24. A Typical File Control Block [ = i-node in Unix]
  25. 25. In-Memory File System Structures
  26. 26. Virtual File Systems <ul><li>Virtual File Systems (VFS) provide an object-oriented way of implementing file systems. </li></ul><ul><li>VFS allows the same system call interface (the API) to be used for different types of file systems. </li></ul><ul><li>The API is to the VFS interface, rather than any specific type of file system. </li></ul>
  27. 27. Schematic View of Virtual File System
  28. 28. Directory Implementation <ul><li>Linear list of file names with pointer to the data blocks. </li></ul><ul><ul><li>simple to program </li></ul></ul><ul><ul><li>time-consuming to execute </li></ul></ul><ul><li>Hash Table – linear list with hash data structure. </li></ul><ul><ul><li>decreases directory search time </li></ul></ul><ul><ul><li>collisions – situations where two file names hash to the same location </li></ul></ul><ul><ul><li>fixed size </li></ul></ul>
  29. 29. Allocation Methods <ul><li>An allocation method refers to how disk blocks are allocated for files: </li></ul><ul><li>Contiguous allocation </li></ul><ul><li>Linked allocation </li></ul><ul><li>Indexed allocation </li></ul>
  30. 30. Contiguous Allocation <ul><li>Each file occupies a set of contiguous blocks on the disk </li></ul><ul><li>Simple – only starting location (block #) and length (number of blocks) are required </li></ul><ul><li>Random access </li></ul><ul><li>Wasteful of space (dynamic storage-allocation problem) </li></ul><ul><li>Files cannot grow </li></ul>
  31. 31. Contiguous Allocation of Disk Space
  32. 32. Linked Allocation
  33. 33. File-Allocation Table
  34. 34. Example of Indexed Allocation
  35. 35. Combined Scheme: UNIX (4K bytes per block)
  36. 36. Free-Space Management <ul><li>Bit vector ( n blocks) </li></ul>… 0 1 2 n-1 bit[ i ] =  0  block[ i ] free 1  block[ i ] occupied Block number calculation (number of bits per word) *(number of 0-value words) + offset of first 1 bit
  37. 37. Free-Space Management (Cont.) <ul><li>Bit map requires extra space </li></ul><ul><ul><li>Example: </li></ul></ul><ul><li>block size = 2 12 bytes </li></ul><ul><li>disk size = 2 30 bytes (1 gigabyte) </li></ul><ul><li>n = 2 30 /2 12 = 2 18 bits (or 32K bytes) </li></ul><ul><li>Easy to get contiguous files </li></ul><ul><li>Linked list (free list) </li></ul><ul><ul><li>Cannot get contiguous space easily </li></ul></ul><ul><ul><li>No waste of space </li></ul></ul><ul><li>Grouping </li></ul><ul><li>Counting </li></ul>
  38. 38. Free-Space Management (Cont.) <ul><li>Need to protect: </li></ul><ul><ul><li>Pointer to free list </li></ul></ul><ul><ul><li>Bit map </li></ul></ul><ul><ul><ul><li>Must be kept on disk </li></ul></ul></ul><ul><ul><ul><li>Copy in memory and disk may differ </li></ul></ul></ul><ul><ul><ul><li>Cannot allow for block[ i ] to have a situation where bit[ i ] = 1 in memory and bit[ i ] = 0 on disk </li></ul></ul></ul><ul><ul><li>Solution: </li></ul></ul><ul><ul><ul><li>Set bit[ i ] = 1 in disk </li></ul></ul></ul><ul><ul><ul><li>Allocate block[ i ] </li></ul></ul></ul><ul><ul><ul><li>Set bit[ i ] = 1 in memory </li></ul></ul></ul>
  39. 39. Directory Implementation <ul><li>Linear list of file names with pointer to the data blocks </li></ul><ul><ul><li>simple to program </li></ul></ul><ul><ul><li>time-consuming to execute </li></ul></ul><ul><li>Hash Table – linear list with hash data structure </li></ul><ul><ul><li>decreases directory search time </li></ul></ul><ul><ul><li>collisions – situations where two file names hash to the same location </li></ul></ul><ul><ul><li>fixed size </li></ul></ul>
  40. 40. Linked Free Space List on Disk
  41. 41. Efficiency and Performance <ul><li>Efficiency dependent on: </li></ul><ul><ul><li>disk allocation and directory algorithms </li></ul></ul><ul><ul><li>types of data kept in file’s directory entry </li></ul></ul><ul><li>Performance </li></ul><ul><ul><li>disk cache – separate section of main memory for frequently used blocks </li></ul></ul><ul><ul><li>free-behind and read-ahead – techniques to optimize sequential access </li></ul></ul><ul><ul><li>improve PC performance by dedicating section of memory as virtual disk, or RAM disk </li></ul></ul>
  42. 42. Page Cache <ul><li>A page cache caches pages rather than disk blocks using virtual memory techniques </li></ul><ul><li>Memory-mapped I/O uses a page cache </li></ul><ul><li>Routine I/O through the file system uses the buffer (disk) cache </li></ul><ul><li>This leads to the following figure </li></ul>
  43. 43. I/O Without a Unified Buffer Cache
  44. 44. Unified Buffer Cache <ul><li>A unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O </li></ul>
  45. 45. I/O Using a Unified Buffer Cache