Os10 2

632 views
564 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
632
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Os10 2

  1. 1. File System and Input/Output Lecture 10: File Systems 主講人:虞台文
  2. 2. Content <ul><li>Basic Functions of File Management </li></ul><ul><li>Hierarchical Model of a File System </li></ul><ul><li>File Directories </li></ul><ul><ul><li>Hierarchical Directory Organizations </li></ul></ul><ul><ul><li>Operations on Directories </li></ul></ul><ul><ul><li>Implementation of File Directories </li></ul></ul><ul><li>Basic File System </li></ul><ul><ul><li>File Descriptors </li></ul></ul><ul><ul><li>Opening and Closing of Files </li></ul></ul><ul><li>Physical Organization Methods </li></ul><ul><ul><li>Contiguous Organization </li></ul></ul><ul><ul><li>Linked Organization </li></ul></ul><ul><ul><li>Indexed Organization </li></ul></ul><ul><ul><li>Management of Free Storage Space </li></ul></ul><ul><li>Distributed File Systems </li></ul><ul><ul><li>Directory Structures and Sharing </li></ul></ul><ul><ul><li>Semantics of File Sharing </li></ul></ul><ul><li>Implementing DFS </li></ul>
  3. 3. File System and Input/Output Lecture 10: File Systems Basic Functions of File Management
  4. 4. Basic Functions of FS <ul><li>Present logical or abstract view of files and directories </li></ul><ul><ul><li>Hide complexity of hardware devices </li></ul></ul><ul><li>Facilitate efficient use of storage devices </li></ul><ul><ul><li>Optimize access , e.g., to disk </li></ul></ul><ul><li>Support sharing </li></ul><ul><ul><li>Provide protection </li></ul></ul>
  5. 5. File System and Input/Output Lecture 10: File Systems Hierarchical Model of a File System
  6. 6. Hierarchical Model of FS
  7. 7. Hierarchical Model of FS
  8. 8. The File System <ul><li>Presents the abstract or logical view of files to users </li></ul><ul><li>Defines a set of operations for file/directory manipulation </li></ul>File manipulation
  9. 9. The File System Map logical name to unique Id, file descriptor
  10. 10. The File System <ul><li>Open/close files </li></ul><ul><li>access right verification </li></ul><ul><li>maintain open file tables (OFT) </li></ul>
  11. 11. The File System <ul><li>Map file data to logical blocks on the device </li></ul><ul><li>Main memory buffer allocation/deallocation </li></ul><ul><li>Keep track the mapping btw memory buffer </li></ul><ul><li>and data blocks </li></ul>0 1 2 3 
  12. 12. File Names and Types <ul><li>Valid name </li></ul><ul><ul><li>Number of characters </li></ul></ul><ul><ul><li>Lower vs upper case </li></ul></ul><ul><li>Extension </li></ul><ul><ul><li>Tied to type of file </li></ul></ul><ul><ul><li>Used by applications </li></ul></ul><ul><li>File type recorded in header </li></ul><ul><ul><li>Cannot be changed (even when extension changes) </li></ul></ul><ul><ul><li>Basic types: text, object, load file; directory </li></ul></ul><ul><ul><li>Application-specific types, e.g., .doc, .ps, .html </li></ul></ul><ul><li>File names and types </li></ul><ul><li>Logical file organization </li></ul><ul><li>Other file attributes </li></ul><ul><li>Operations on files </li></ul>
  13. 13. File Names and Types <ul><li>Valid name </li></ul><ul><ul><li>Number of characters </li></ul></ul><ul><ul><li>Lower vs upper case </li></ul></ul><ul><li>Extension </li></ul><ul><ul><li>Tied to type of file </li></ul></ul><ul><ul><li>Used by applications </li></ul></ul><ul><li>File type recorded in header </li></ul><ul><ul><li>Cannot be changed (even when extension changes) </li></ul></ul><ul><ul><li>Basic types: text, object, load file; directory </li></ul></ul><ul><ul><li>Application-specific types, e.g., .doc, .ps, .html </li></ul></ul><ul><li>File names and types </li></ul><ul><li>Logical file organization </li></ul><ul><li>Other file attributes </li></ul><ul><li>Operations on files </li></ul>
  14. 14. Logical File Organization <ul><li>Logical Records </li></ul><ul><ul><li>Fixed or variable-size records </li></ul></ul><ul><ul><li>Keyed or unkeyed records </li></ul></ul><ul><li>File names and types </li></ul><ul><li>Logical file organization </li></ul><ul><li>Other file attributes </li></ul><ul><li>Operations on files </li></ul>a) Fixed Length Record b) Variable Length Record c) Fixed Length with Key d) Variable Length with Key
  15. 15. Logical File Organization <ul><li>Logical Records </li></ul><ul><ul><li>Fixed or variable-size records </li></ul></ul><ul><ul><li>Keyed or unkeyed records </li></ul></ul><ul><li>File names and types </li></ul><ul><li>Logical file organization </li></ul><ul><li>Other file attributes </li></ul><ul><li>Operations on files </li></ul>a) Fixed Length Record b) Variable Length Record c) Fixed Length with Key d) Variable Length with Key
  16. 16. Logical File Organization <ul><li>Record Addressing </li></ul><ul><ul><li>Implicitly (sequential access) </li></ul></ul><ul><ul><li>Explicitly by position or key (direct or random access) </li></ul></ul><ul><li>File names and types </li></ul><ul><li>Logical file organization </li></ul><ul><li>Other file attributes </li></ul><ul><li>Operations on files </li></ul>a) Fixed Length Record b) Variable Length Record c) Fixed Length with Key d) Variable Length with Key
  17. 17. Logical File Organization <ul><li>Memory Mapped File </li></ul><ul><ul><li>Making use of virtual memory system </li></ul></ul><ul><ul><li>Read virtual memory instead of read(i,buf,n) </li></ul></ul><ul><ul><li>Map file contents 0:( n  1) to va :( va + n  1) </li></ul></ul><ul><li>File names and types </li></ul><ul><li>Logical file organization </li></ul><ul><li>Other file attributes </li></ul><ul><li>Operations on files </li></ul>CreateFileMapping
  18. 18. Other file attributes <ul><li>Ownership </li></ul><ul><li>File Size </li></ul><ul><li>File Use </li></ul><ul><ul><li>Time of creation , last access , last modification </li></ul></ul><ul><li>File Disposition </li></ul><ul><ul><li>Permanent or Temporary </li></ul></ul><ul><li>Protection </li></ul><ul><ul><li>Who can access and what type of access </li></ul></ul><ul><li>Location </li></ul><ul><li>File names and types </li></ul><ul><li>Logical file organization </li></ul><ul><li>Other file attributes </li></ul><ul><li>Operations on files </li></ul>
  19. 19. Operations on files <ul><li>Create/Delete </li></ul><ul><ul><li>Create/delete file descriptor </li></ul></ul><ul><ul><li>Modify directory </li></ul></ul><ul><li>Open/Close </li></ul><ul><ul><li>Open  Setup open file table (“OFT”) & buffer </li></ul></ul><ul><ul><li>Close  Reverse the effect of open </li></ul></ul><ul><li>Read/Write (sequential or direct) </li></ul><ul><ul><li>Modify file descriptor </li></ul></ul><ul><ul><li>Transfer data between disk and buffer </li></ul></ul><ul><li>Seek/Rewind </li></ul><ul><ul><li>Modify open file table </li></ul></ul><ul><li>File names and types </li></ul><ul><li>Logical file organization </li></ul><ul><li>Other file attributes </li></ul><ul><li>Operations on files </li></ul>
  20. 20. File System and Input/Output Lecture 10: File Systems File Directories
  21. 21. File Directories <ul><li>Every file directory is itself a file . </li></ul><ul><li>It records the information about other files , including possibly other directories . </li></ul>
  22. 22. Hierarchical Directory Organizations <ul><li>Tree -Structured </li></ul><ul><li>DAG -Structured </li></ul>
  23. 23. Tree-Structured Directories <ul><li>Simple search , insert , delete operations </li></ul><ul><li>Sharing is asymmetric (only one parent, owner) </li></ul>
  24. 24. DAG-Structured Directories
  25. 25. DAG-Structured Directories a p p a e File F8 shared
  26. 26. DAG-Structured Directories a z Directory D6 shared
  27. 27. DAG-Structured Directories <ul><li>Sharing is symmetric , but </li></ul><ul><li>What are semantics of delete ? </li></ul><ul><ul><li>Any parent can remove file. </li></ul></ul><ul><ul><li>Only last parent can remove it. Need reference count </li></ul></ul>
  28. 28. DAG-Structured Directories <ul><li>Must prevent cycles </li></ul>
  29. 29. DAG-Structured Directories <ul><li>Must prevent cycles </li></ul><ul><li>If cycles are allowed: </li></ul><ul><ul><li>Search is difficult (infinite loops) </li></ul></ul><ul><ul><li>Deletion needs garbage collection (reference count not enough) </li></ul></ul>
  30. 30. DAG-Structured Directories How to do rd a , i.e., remove directory D2 ? Deletion needs garbage collection (reference count not enough)  <ul><li>Delete entry a of root. </li></ul><ul><li>Start from root to mark all reachable nodes. </li></ul><ul><li>Delete all unmarked nodes. </li></ul>marked unmarked  Very inefficient
  31. 31. Symbolic Links Main directory is tree-structured . Sharing via symbolic link  allow cycles . F 4 a z b c n a p l r t e f h k p n m k q b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 8 F 9 F 10 F 11 Root
  32. 32. Symbolic Links <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>Root F 4 a z b c n a p l r t e f h k p n m k q b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 8 F 9 F 10 F 11
  33. 33. Symbolic Links F 4 a z b c n a p l r t e f h k p n m k q b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 8 F 9 F 10 F 11 <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>del p Root
  34. 34. Symbolic Links F 4 a z b c n a l r t e f h k p n m k q b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 8 F 9 F 10 F 11 <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>del p Root
  35. 35. Symbolic Links F 4 a z b c n a p l r t e f h k p n m k q b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 8 F 9 F 10 F 11 <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>del a e Root
  36. 36. Symbolic Links F 4 a z b c n a l r t f h k n m k q b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 9 F 10 F 11 <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>del a e Phantom References Root p p
  37. 37. Symbolic Links F 4 a z b c n a p l r t e f h k p n m k q b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 8 F 9 F 10 F 11 <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>rd a Root
  38. 38. Symbolic Links b c n a D 1 D 3 F 1 F 2 F 3 <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>rd a Phantom References Root z p
  39. 39. Symbolic Links F 4 a z b c n a p l r t e f h k p n m k q b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 8 F 9 F 10 F 11 <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>rd almq Root
  40. 40. Symbolic Links F 4 a z b c n a p l r t e f h k p n m k b c D 1 D 2 D 3 D 4 D 5 D 6 D 7 F 1 F 2 F 3 F 5 F 6 F 7 F 8 F 9 F 10 F 11 <ul><ul><li>For read / write access: </li></ul></ul><ul><ul><li>Symbolic link is the same as actual link </li></ul></ul><ul><ul><li>For deletion : </li></ul></ul><ul><ul><li>Only symbolic link is deleted </li></ul></ul>del almq Root
  41. 41. File/Directory Naming: Path Names <ul><li>Concatenated local names with delimiter: </li></ul><ul><ul><li>. or / or </li></ul></ul><ul><li>Absolute path name: start with root </li></ul><ul><ul><li>/ or </li></ul></ul><ul><li>Relative path name: Start with current directory </li></ul><ul><ul><li>. </li></ul></ul><ul><li>Notation to move “upward” </li></ul><ul><ul><li>.. </li></ul></ul>
  42. 42. Operations on File Directories <ul><li>Create/delete </li></ul><ul><li>List </li></ul><ul><ul><li>sorting, wild cards, recursion, information displayed </li></ul></ul><ul><li>Change (current, working, default) directory </li></ul><ul><ul><li>path name, home directory (default) </li></ul></ul><ul><li>Move </li></ul><ul><li>Rename </li></ul><ul><li>Change protection </li></ul><ul><li>Create/delete link (symbolic) </li></ul><ul><li>Find/search routines </li></ul>Original Copied
  43. 43. Implementation of File Directories <ul><li>What information to keep in each entry? </li></ul><ul><ul><li>Only symbolic name and pointer to descriptor </li></ul></ul><ul><ul><ul><li>Needs an extra disk access to descriptor </li></ul></ul></ul><ul><ul><li>All descriptive information </li></ul></ul><ul><ul><ul><li>Directory can become very large </li></ul></ul></ul><ul><li>How to organize entries within directory? </li></ul><ul><ul><li>Fixed-size array of slots or a linked list </li></ul></ul><ul><ul><ul><li>Easy insertion / deletion </li></ul></ul></ul><ul><ul><ul><li>Search is sequential </li></ul></ul></ul><ul><ul><li>Hash table </li></ul></ul><ul><ul><ul><li>Easy insertion / deletion/search </li></ul></ul></ul><ul><ul><ul><li>Hard to determine the table size </li></ul></ul></ul><ul><ul><li>B-tree (balanced, but sequential access can be slow ) </li></ul></ul><ul><ul><li>B + -tree (balanced and with good sequential access ) </li></ul></ul>
  44. 44. File System and Input/Output Lecture 10: File Systems Basic File System
  45. 45. Basic File System <ul><li>Open/close files </li></ul><ul><li>access right verification </li></ul><ul><li>maintain open file tables (OFT) </li></ul>
  46. 46. Open/Close Files <ul><li>Retrieve and set up descriptive information for efficient access . </li></ul>FILE * fopen ( const char *filename, const char *mode ); int fclose ( FILE * stream ); size_t fread ( void *buffer, size_t size, size_t count, FILE * stream ); size_t fwrite ( const void *buffer, size_t size, size_t count, FILE * stream ); Block-oriented I/O in UNIX Low-level I/O in Windows Stream I/O in Windows
  47. 47. File Descriptors (Unix i-node) <ul><li>Owner id </li></ul><ul><li>File type </li></ul><ul><li>Protection information </li></ul><ul><li>Mapping to physical disk blocks </li></ul><ul><li>Time of creation, last use, last modification </li></ul><ul><li>Reference counter </li></ul>
  48. 48. Open File Table <ul><li>To keep track the currently open files of the system. </li></ul><ul><li>Managed by the open/clos e functions </li></ul>
  49. 49. Open <ul><li>Verify access rights </li></ul><ul><li>Allocate OFT entry </li></ul><ul><li>Allocate read/write buffers </li></ul><ul><li>Fill in OFT entry </li></ul><ul><ul><li>Initialization (e.g., current position) </li></ul></ul><ul><ul><li>Information from descriptor (e.g. file length, disk location) </li></ul></ul><ul><ul><li>Pointers to allocated buffers </li></ul></ul><ul><li>Return OFT index </li></ul>
  50. 50. Close <ul><li>Flush modified buffers to disk </li></ul><ul><li>Release buffers </li></ul><ul><li>Update file descriptor , e.g., </li></ul><ul><ul><li>file length </li></ul></ul><ul><ul><li>disk location </li></ul></ul><ul><ul><li>usage information (e.g., date of last access/modification) </li></ul></ul><ul><li>Free OFT entry </li></ul>
  51. 51. Example: Open File Tables in Unix <ul><li>Unbuffered access </li></ul><ul><li> fd= open (name,rw,…) stat= read (fd,mem,n) stat= write (fd,mem,n ) </li></ul><ul><li>Buffered access </li></ul><ul><li> fp= fopen (name,rwa) c= readc (fp ) </li></ul>
  52. 52. Example: Open File Tables in Unix <ul><li>Unbuffered access </li></ul><ul><li> fd= open (name,rw,…) stat= read (fd,mem,n) stat= write (fd,mem,n ) </li></ul><ul><li>Buffered access </li></ul><ul><li> fp= fopen (name,rwa) c= readc (fp ) </li></ul>
  53. 53. File System and Input/Output Lecture 10: File Systems Physical Organization Methods
  54. 54. Main Secondary Storage Media tapes disks From the file system of views, they all considered as a device with 1D sequence of logical blocks by numbering them from 0 to n  1 .
  55. 55. Device Organization Methods <ul><li>Contiguous organization </li></ul><ul><li>Linked Organization </li></ul><ul><li>Indexed Organization </li></ul>
  56. 56. Contiguous Organization <ul><li>Simple implementation </li></ul><ul><li>Fast sequential access (minimal arm movement) </li></ul><ul><li>Insert / delete is difficult </li></ul><ul><li>How much space to allocate initially </li></ul><ul><li>External fragmentation </li></ul><ul><li>Contiguous organization </li></ul><ul><li>Linked Organization </li></ul><ul><li>Indexed Organization </li></ul>Device Organization Methods:
  57. 57. Linked Organization <ul><li>Simple insert / delete , no external fragmentation </li></ul><ul><li>Sequential access less efficient (seek latency) </li></ul><ul><li>Direct access not possible </li></ul><ul><li>Poor reliability (when chain breaks) </li></ul><ul><li>Contiguous organization </li></ul><ul><li>Linked Organization </li></ul><ul><li>Indexed Organization </li></ul>Device Organization Methods:
  58. 58. Linked Organization Variations <ul><li>Variation 1 : Keep pointers segregated </li></ul><ul><li>Variation 2 : Link sequences of adjacent blocks </li></ul><ul><li>Contiguous organization </li></ul><ul><li>Linked Organization </li></ul><ul><li>Indexed Organization </li></ul>Device Organization Methods:
  59. 59. Indexed Organization Variations <ul><li>Multi-level index hierarchy </li></ul><ul><ul><li>Primary index points to secondary indices </li></ul></ul><ul><ul><li>Problem: number of disk accesses increases with depth of hierarchy </li></ul></ul><ul><li>Incremental indexing </li></ul><ul><ul><li>Fixed number of entries at top-level index </li></ul></ul><ul><ul><li>When insufficient , allocate additional index levels </li></ul></ul><ul><ul><li>E.g., Unix  3-level expansion (see next) </li></ul></ul><ul><li>Contiguous organization </li></ul><ul><li>Linked Organization </li></ul><ul><li>Indexed Organization </li></ul>Device Organization Methods:
  60. 60. Example: Unix -- 3-Level Expansion <ul><li>Contiguous organization </li></ul><ul><li>Linked Organization </li></ul><ul><li>Indexed Organization </li></ul>Device Organization Methods: Incremental indexing
  61. 61. Free Storage Space Management <ul><li>Linked list organization </li></ul><ul><ul><li>Linking individual blocks  inefficient: </li></ul></ul><ul><ul><ul><li>Multiple of blocks are allocated/released at a time </li></ul></ul></ul><ul><ul><ul><li>No block clustering to minimize seek operations </li></ul></ul></ul><ul><ul><li>Linking groups of consecutive blocks </li></ul></ul><ul><li>Bitmap organization </li></ul><ul><ul><li>Analogous to main memory </li></ul></ul>Similar to main memory management .
  62. 62. File System and Input/Output Lecture 10: File Systems Distributed File Systems
  63. 63. Principles of Distributed File Systems <ul><li>Sharing </li></ul><ul><ul><li>to support sharing of files and resources in the form of persistent storage over a network . </li></ul></ul><ul><li>Transparencies </li></ul><ul><ul><li>to present a unifying view of all files to the user </li></ul></ul><ul><ul><li>Location transparent </li></ul></ul><ul><ul><li>Location independent </li></ul></ul>
  64. 64. Directory structures <ul><li>Directory structures differentiated by: </li></ul><ul><li>Global vs. Local naming: </li></ul><ul><ul><li>Single global structure or different for each user ? </li></ul></ul><ul><li>Location transparency: </li></ul><ul><ul><li>Does the path name reveal anything about machine or server ? </li></ul></ul><ul><li>Location independence: </li></ul><ul><ul><li>When a file moves between machines, does its path name change ? </li></ul></ul>
  65. 65. Global Directory Structure Server S 1 Server S 2 usr mp    u 1       root directory (/)    usr loc    u 2          root directory (/) x y S 1 S 2
  66. 66. Global Directory Structure Server S 1 Server S 2 How to build a ` global ’ directory? usr loc    u 2          root directory (/) x y usr mp    u 1       root directory (/)   
  67. 67. Global Directory Structure Server S 1 Server S 2 How to build a ` global ’ directory? local root directory (/) local root directory (/) Global root directory S 1 S 2 usr mp    u 1       root directory (/)    usr loc    u 2          root directory (/) x y
  68. 68. How to Name a File/Directory? usr mp    u 1       root directory (/)    Server S 1 Server S 2 root directory (/) local root directory (/) local root directory (/) Global root directory S 1 S 2 /S1/usr/u1 <ul><li>Problem with “Combine under new common root:” </li></ul><ul><ul><li>Using “ / ” for new root invalidates existing local names . </li></ul></ul>usr loc    u 2          root directory (/) x y
  69. 69. How to Name a File/Directory? <ul><li>Solution (Unix United): </li></ul><ul><ul><li>Use “ / ” for local root </li></ul></ul><ul><ul><li>Use “ .. ” to move to new root </li></ul></ul><ul><li>Names are not location transparent </li></ul>usr mp    u 1       root directory (/)    Server S 1 Server S 2 root directory (/) local root directory (/) local root directory (/) Global root directory S 1 S 2 /S1/usr/u1 <ul><li>Problem with “Combine under new common root:” </li></ul><ul><ul><li>Using “ / ” for new root invalidates existing local names . </li></ul></ul>usr loc    u 2          root directory (/) x y

×