Successfully reported this slideshow.

Sketch of the ZXFS

795 views

Published on

(a course project solution) My design of a tag-based (instead of pathname-based) file system

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Sketch of the ZXFS

  1. 1. Sketch of the ZXFSBy @ZxMYShttp://www.slideshare.net/ZxMYS
  2. 2. What’s it• ZXFS - Zhu Xiao’s File System• A file system that satisfy all Ben’s requirements• Similar to the design of the UNIX file system• Implementable, Efficient and Simple
  3. 3. LAYERSTHE BASICS.
  4. 4. Layers in ZXFSLayer ProposeSymbolic link layer Integrate multiple file systems with symbolic links.High level scope (HLS) layer Provide a root for the naming user-oriented names hierarchies.Scope layer Organize files into naming hierarchies.Tag layer Provide human-oriented tags for machine-user interface files.Inode Number layer Provide machine-oriented names for files. machine-oriented namesFile layer Organize blocks into files.Block layer Identify disk blocks.
  5. 5. Layers in ZXFSLayer ProposeDue to compatible design in lower layers, we can take this layer directly.Symbolic link layer Integrate multiple file systems with symbolic links.High level scope (HLS) layer Provide a root for the naming user-oriented names hierarchies.Scope layer Organize files into naming hierarchies.Tag layer Provide human-oriented tags for machine-user interface files.Inode Number layer Provide machine-oriented names for files. machine-oriented namesFile layer Organize blocks into files.Block layer Identify disk blocks.
  6. 6. Layers in ZXFSLayer ProposeSymbolic link layer Integrate multiple file systems with symbolic links.Mainly use these three layers to achieve the goal .High level scope (HLS) layer Provide a root for the naming user-oriented names hierarchies.Scope layer Organize files into naming hierarchies.Tag layer Provide human-oriented tags for machine-user interface files.Inode Number layer Provide machine-oriented names for files. machine-oriented namesFile layer Organize blocks into files.Block layer Identify disk blocks.
  7. 7. Layers in ZXFSLayer ProposeSymbolic link layer Integrate multiple file systems with symbolic links.High level scope (HLS) layer Provide a root for the naming user-oriented names hierarchies.Scope layer Organize files into naming hierarchies.Tag layer Provide human-oriented tags forNeed some modification in implementation machine-user interface files.Inode Number layer Provide machine-oriented names for files. machine-oriented namesFile layer Organize blocks into files.Block layer Identify disk blocks.
  8. 8. Layers in ZXFSLayer ProposeSymbolic link layer Integrate multiple file systems with symbolic links.High level scope (HLS) layer Provide a root for the naming user-oriented names hierarchies.Scope layer Organize files into naming hierarchies.Tag layer Provide human-oriented tags for machine-user interface files.Inode Number layer Provide machine-oriented names for files.Take them directly machine-oriented namesFile layer Organize blocks into files.Block layer Identify disk blocks.
  9. 9. Layers in ZXFS - the Tag layer• Definition:Tag: Some (name, value) pairs attached to files• All files are located (in the final step) by tags, thus the tag layer has similar functionality like old file name layer.• A file can have multi-tags, but must has at least one tag, otherwise it should be deleted.• Some traditional metadata, such as ctime, mime and atime, are dealt as file.
  10. 10. Layers in ZXFS - the Scope Layer• Definition:Scope: A set of something. In this layer we discuss‘scope for file’, which is a set of files, and ‘scope fortag’, a set of files tagged with a certain tag.• Each tag has it’s scope.• Use scopes to string (group) files, like directory.• Scope can be static or dynamic. Can present them in memory.• Device store scopes for all tags on that device.
  11. 11. Layers in ZXFS - the High Level Scope (HLS) Layer• Definition:HLS: ‘scope for scope’• A set of scopes, providing root for scopes• Like ‘/’ in traditional UNIX FS• Each device has it’s scope. When system start up, system merge them into memory and get an overall HLS
  12. 12. Layers in ZXFS - Path Name Stimulating with these 3 layers• To be more compatible with traditional UNIX FS and enable symbolic links.• Consider following path:/(mount point of ZXFS)/tag_expression1/tag_expression2/...../tag_expressionN• Tag Expression is some expression used to locate file using tag, like SQL or XPath• latter N-1 ‘/’ perform AND logic here
  13. 13. Layers in ZXFS - Path Name Stimulating with these 3 layers• Example/mnt/zxfs1/`ctime` greater than #2011-4-19#/`atime`equal to #2011-4-20#/• Looks strange but can work• Resulting in a scope contains all files have tag ctime and atime satisfying the expression• If this scope has multiple files, return a directory (or something like it)• Else, return the file or null• NOTE: THE DESIGN OF TAG EXPRESSION IS NOT A PART OF THE DESIGN OF ZXFS. SO ONLY MENTION IT HERE.
  14. 14. EXAMPLE LAYER IMPLEMENTATIONLET’S CALL IT ZXFS 0.1
  15. 15. Sketch
  16. 16. SketchStored in diskexactly like file data
  17. 17. Basic Knowledge – B+ tree• a B+ tree or B plus tree is a type of tree which represents sorted data in a way that allows for efficient insertion, retrieval and removal of records• The primary value of a B+ tree is in storing data for efficient retrieval in a block-oriented storage context—in particular, file systems. Because it can reduce the number of I/O operations required to find an element in the tree.
  18. 18. Basic Data Structure - inodestruct inode integer type, refcnt, filesize, userid, groupid, mode //type = FILE_TYPE or SCOPE_TYPE integer block_numbers_for_file[N] //blocks used to save file union struct integer tagsize //size of the tags integer block_numbers_for_tags[M-1] //if type==FILE_TYPE. blocks used to //save tags for this file integer rest_block_numbers_for_file[M] //if type== SCOPE_TYPE or other type that have no //tags, the rest space of this inode block can be //used to store more inode number for filesChange inode structure into this, to implement tags and scopes
  19. 19. Basic Data Structure – tag_datastruct tag_data integer size //indicate the size of this tag_data integer inode_number_for_scope //pointer to the scope fot this tag string tag_value Every file has a B+ tree, in which (key, value)=(tag name, tag_data) And this B+ tree is called tag data of this file
  20. 20. Basic Data Structure – scope_datastruct scope_data string tag_name //tag name of this scope B_plus_tree files //B+ tree which store info about file tagged with this tag The scope_data represent scope. B+ tree here has (key, value) = (tag value, file inode number) (in device) or (key, value) = (tag value, (deviceID, file inode number)) (in memory)
  21. 21. Basic Data StructureHLS is a B+ tree with(key, value) = (tag name, inode of scope for that tag)(in device)Or(key, value) = (tag name, (deviceID, inode in that device ofscope for that tag))(in memory)
  22. 22. Sketch, again
  23. 23. THE ZXFS APIVERY EASY IF YOU HAVE A NICE LAYER DESIGN
  24. 24. Outline Of API• Most UNIX API can be implemented on ZXFS• ALL Ben’s API can be implemented on ZXFS
  25. 25. Part of API description tableAPI descriptionsearch(device ID, tag name list, destination Search for files that have tagged with given tag. Device ID=0 indicatesscope ID) overall HLS of systemsearch(device ID or scope ID, tag expression Search for files satisfying given tag expressions. ID=0 indicates overalllist, destination scope ID) HLS of systemcreate(device ID) Return a file ID for a new file on given device.delete(device ID, file ID) Remove all tags on that file, and delete that file.read/write (device ID, file ID, offset, buf, Read/Write filelength)list(scope ID) Returns a list of (device ID, file ID) tuples for files in that scope. Implement by Iterating B+ tree.mkscope() Creates an empty scope and return it’s ID. Only for scopes in memory.merge_scope(source scope ID, destination Merge two scopes. Destination scope can only be in memoryscope ID)tag_add/tag_remove (device ID, file ID, tag Add/Remove tag on a file.name)tag_get(device ID, file ID) List all tags on that file. Implement by reading tagdata of that file.get_ HLS (device ID) Get HLS. When the device ID is 0, get the system (overall) HLSdevice_list() Returns the list of currently plugged-in devices
  26. 26. Some Of API Implementation under ZXFS 0.1• SEARCH (TAG NAME LIST)Just do search on B+ tree of HLS. If user gives adevice ID, use HLS of that tag• SEARCH (TAG EXPRESSION)Use a tag expression interpreter to interpret theexpression first, and then do search on HLS, findcorresponding scopes for tags, do another searchon them. Still, the search is performed on B+ tree(thus very quick)
  27. 27. Some Of API Implementation under ZXFS 0.1• CREATEJust find an empty inode and tag it with basictags like atime/ctime and mtime, updatecorresponding scopes• DELETECall tag_remove to remove all tags on that file.The last remove action will cause the file beremoved.
  28. 28. Some Of API Implementation under ZXFS 0.1• TAG_ADD/TAG_REMOVEBy changing tag data of that file and updatecorresponding scope (including HLS if necessary).If a new tag name is introduced, make a newscope for it on the device. If all tag of a file isremoved, remove that file.
  29. 29. ANALYSISTHE TIME TO CHECK UP
  30. 30. Performance?• The performance can’t be very bad since we use B+ tree for searching, creating or removing tag.• Tag data has pointers back to scope, so updating tag_value or the removal of a tag is fast.
  31. 31. A Trade-off• If we want to change a tag’s name, all file tagged with that must be modified as well as some coherence should be considered.• However, put tag name inside tag data enable us to find a certain tag quickly.• We assume that finding a tag for a file happens more often than changing name of a tag.
  32. 32. Another Trade-off• Storing tag name in every tag data require large space. – Assume user will NOT use long tag name for many files.• B+ tree is very large, comparing to other data structure, in memory.• Duplicate pointers to file in scopes for ctime/mtime/atime, etc.SOLUTION: ZXFS prefer time to space!
  33. 33. OTHER THINGSMAYBE WE HAVE FORGOTTEN SOMETHING?
  34. 34. The practicability of tag-based FS• If the user is hardworking in adding tags for files, It’s OK.• What if user is lazy? – We need advanced AI to help, or do it by itself, to add tags for files, according to the content of that file. – In near future? Maybe.
  35. 35. THE ENDDO YOU HAVE QUESTIONS?

×