Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AFS introduction


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

AFS introduction

  1. 1. Introduction to OpenAFSFabrizio Manfred Furuholmen
  2. 2. Agenda  Introduction  Architecture  Client  Administration tasks  Setup your Cell 2 16/02/2012
  3. 3. Introduction What is a Distributed File system ? “A distributed file system takes advantage of the interconnected nature of the network by storing files on more than one computer in the network and making them accessible to all of them..” 3 16/02/2012
  4. 4. Introduction Andrew File System Andrew File System is a distributed file system designed to:  handle terabytes of data  handle thousands of users  working in WAN environment 4
  5. 5. Introduction AFS is25yearsold ! 5
  6. 6. Introduction Briefhistoryof a AFS 1983 Andrew Project started at Carnegie Mellon University (CMU) 1987 Coda research work begun (based on AFS) 1988 First use of AFS version 3 (First use of AFS outside CMU) 1988 Institutional File System project at University of Michigan 1989 Transarc Corporation founded to commercialize AFS 1993 Arla project started at KungligaTekniskaHögskolan 1998 Transarc Corporation becomes wholly owned subsidiary of IBM 2000 IBM releases OpenAFS as OpenSource (IBM License) 2000 OpenAFS release version 1.0 based on Transarc 3.6 2001 OpenAFS release version 1.2 first release with better support of new operating system and fix several memory leak 2005 OpenAFS release version 1.4 with a lot of new feature 2005 AFS was discontinued from IBM 2008 U.S. Department of Energy Funds OpenAFS Development 2010 OpenAFS release version 1.6 (?) 6
  7. 7. Benefits of using AFS  Location independence User does not need to know which fileserver holds the file, the user only needs to know the pathname of a file.  Scalability An architectural goal of the AFS designers was client/server ratios of 114.000:1 A ratio of 2000:1 has been successfully exceeded at some sites.  Security AFS makes use of Kerberos for mutual authentication, both the service provider and the requester prove their identities AFS uses access control list (ACLs) to enable users to restrict access to their own directories, users can also create groups AFS Federation with inter cell grant  Uniform Namespace No matter where users are logged in, they see the same files  Replicates AFS Volumes Frequently accessed data can be read-only replicated on several servers (rw with osd version). Client will access the closest volume copy or load balance from a different replica 7
  8. 8. Benefits of using AFS  Improved robustness to server crash Clients maintain Local copies of accessed files, replicated read-only volumes on alternate fileservers can satisfy requests for a files  Wide Area Network AFS communications protocols is optimized for WAN. Retransmitting only the single bad packet in a batch of packets (RPC)  Improve system management capability Configuration changes can be made from any client in the AFS cell AFS volumes can move from one server to another without users noticing it  Operating system independent AFS client software runs on many systems (12 platforms) 8
  9. 9. Elements Cell •Cell is collection of file servers and workstation •The directories under /afs are cells, unique tree •Fileserver contains volumes Volumes •Volumes are "containers" or sets of related files and directories •Have size limit •3 type rw, ro, backup Mount Point Directory Server A •Access to a volume is provided through a mount point Server C •A mount point is just like a static directory Server A+B 9
  10. 10. Architecture 10 16/02/2012
  11. 11. Consistency “..That notion of callbacks gives OpenAFS a much stronger consistency guarantee than most other distributed filesystems.” Cache Manager Client-side caching lets clients access data from their local cache without going across the network for every access. Callbacks OpenAFS uses callbacks, which are a promise from the file server to the client that if the file changes, the server will contact the client to tell the client to invalidate the cached contents. 11
  12. 12. Write operation Example write operation client side 1. create file rpc 2. write chunks into cache (interrupted by store_data RPC) 3. read from cache 4. transfer over network 5. write to /vicepXX 12
  13. 13. Write operation Example write operation server side 1 Create file 2 Check metadata, permission, quota and return file path 3 write file into /vicepXX 4 Update meta data on server 5 Update db 13
  14. 14. Client side Client 14
  15. 15. Installation Supported clients AIX 5 and 6 (though 6.3)  FreeBSD 7, 8 and current  HP-UX 11.0, 11i v1 and v2 Irix 6.5  Linux 2.2, 2.4, 2.6 (ia32, ia64, x86_64, ppc, ppc64, arm, sparc, sparc64, s390, s390x) MacOS 10.3, 10.4, 10.5, 10.6 (including 64 bit). OpenBSD 4.4, 4.5, 4.6, 4.7.  Solaris 2.6, 7, 8, 9, 10, 11 (and OpenSolaris)  Also Windows ... 15
  16. 16. Installation ConfigurationDownload and install client package and kernel module Configure krb5 if you use it Configure AFS Files ThisCell : the name of your cell CellServDB : cell list ( of the world) cacheinfo : cache configuration (dimension and location) 16
  17. 17. AuthenticationAuthentication Kerberos 5 kinit, retrieve a kerberos ticket aklog, convert the krb5 ticket in afs token Authentication Kaserver klog, retrieve a afs token Token operations Ticket cache: FILE:/tmp/krb5cc_0 klist, list tikets Default principal: manfred/admin@FARM.ZEROPIU.COM Valid starting Expires Service principal 08/16/10 16:03:46 08/17/10 16:03:46 krbtgt/FARM.ZEROPIU.COM@FARM.ZEROPIU.COM 08/16/10 16:03:54 08/17/10 16:03:46 afs/  tokens, list afs token Tokens held by the Cache Manager: kdestroy, ticket destroy Users (AFS ID 15) tokens for [Expires Aug 17 16:03] unlog, token destroyDon’t Forget  Credentials expire after some time  AFS service ticket is in the kernel memory 17
  18. 18. Access rights ACLs are only for directories ! (Files soon)  ACL inheritance, AFS copies ACL on a parent directory over to a new subdirectory at the time of creation  ACL awareness, not many commands are aware of ACLs (copy) ACL Permission lookup (l) List contents of directory insert (i) Add Files or directories delete (d) Delete entries in directory administer (a) Manipulate ACL for directory read (r) Read file content, query file status write (r) Write file content, change Unix permissions look (k) Full file advisory lock 18
  19. 19. Server side Servers 19
  20. 20. Architecture 20 16/02/2012
  21. 21. Process Server Process Function bosserver Basic OverSeer Server fileserver Serves the files volserver Serves volume data vlserver Volume location server ptserver Protection server buserver Backup server upserver Update server upclient Update client 21
  22. 22. Architecture Beolink.orgAFS ServersTypes Fileserver machine  file storage Database server machine  File and Volume localization  Groups administration  Authentication provider  Backup database Binary distribution  Master server for afs binary (specific architecture) System control machine  Time server  AFS configuration master
  23. 23. Commands VOS PTS Administration FS Commands BOS 23
  24. 24. FS Beolink.orgfs: Commands are:apropos search by help textcheckservers check local cells servers fs commandcheckvolumes check volumeID/name mappingscleanacl clean up access control listcopyacldiskfree copy access control list show server disk space usage  Cache management administrationexamineexportafs display file/volume status enable/disable translators to AFS  Quota managementflushflushmount flush file from cache flush mount symlink from cache ACLs managementflushvolume flush all data in volumegetcacheparms get cache usage info  Mount management on the AFS pathgetcalleraccess list callers accessgetcellstatus get cell statusgetclientaddrs get client network interface addressesgetcrypt get cache manager encryption flaggetfid get fid for file(s) setcachesize set cache sizegetserverprefs get server ranks setcbaddr configure callback connection addresshelp get help on commandslistacl list access control list setcell set cell statuslistaliases list configured cell aliases setclientaddrs set client network interface addresseslistcells list configured cells setcrypt set cache manager encryption flaglistquota list volume quota setquota set volume quotalsmount list mount point setserverprefs set server ranksmessages control Cache Manager messagesmkmount make mount point setvol set volume statusnewalias configure new cell alias storebehind store to server after file closenewcell configure new cell sysname get/set sysname (i.e. @sys) valuequota show volume quota usage uuid manage the UUID for the cache managerrmmount remove mount point whereis list files locationrxstatpeer Manage per peer RX statisticsrxstatproc Manage per process RX statistics whichcell list files cellsetacl set access control list wscell list workstations cellsetcachesize set cache size 24
  25. 25. Administration BOS Command  Process creation  Process administration (start, stop, status ...)  Manage Users Administrator for process  Volume check 25
  26. 26. Administration VOS Command  Create volume  Volume Replication  Volume Information  Move volume DON’T FORGET THERE ARE DIFFERENT PATH FOR RW and R0 RW = /afs/cell/.mount_point RO = /afs/cell/mount_point 26
  27. 27. Administration Beolink.orgPTS Command Create id for users and groups  Users have a positive number  Groups use negative number Management of Group/User membership Management metadata (group quota, flags)IMPORTANT Users can create their own groups Special groups system:anyuser system:authuser system:administrators 27
  28. 28. Limits Limits 28
  29. 29. Limits General Limits OpenAFS can support a maximum of 114.000 clients per server tmpfs no work as AFS Cache, (ramdisk work)  Max 255 partition per server (/vicepa-/vicepiv), no limits in partition size  Max 4,294,967,295 volumes per partition (this a limit of VLDB),  Max file limit per directory is 64,000 files (less than 16 characters).  Windows Limits  No integration on Microsoft DFS  No native implementation 29
  30. 30. Limits AFS does not allow certain type of files:  Pipes  Device files  Socket  AFS cannot do byte range locking on all platform  Client has working byte rage locks  Full file locks on the server ACLs works on directories not files (yes)  AFS does not support mandatory file locks  DES Encryption for file transport 30
  31. 31. Weaknesses AFS is not so well suited for these situation  No reuse of read data  Access to file larger than cache  Mostly write access  Larger numbers of directory entry changes from multiple clients 31
  32. 32. Introduction Full circle Storage is cheap. Managing storage is more expensive.Wide access to data is still critical.Today and into the future. 32
  33. 33. Thanksto... Alf Watchsmann for usage of “Introduction to AFS and its Best Practices” Please read the original presentation for a complete overview For more information read Documentation on Other presentation are available on 33
  34. 34. I look forwardto meeting you… XVII European AFS meeting 2011 HAMBURG – GERMANY Who should attend:  Everyone interested in deploying a globally accessible file system  Everyone interested in learning more about real world usage of Kerberos authentication in single realm and federated single sign-on environments  Everyone who wants to share their knowledge and experience with other members of the AFS and Kerberos communities  Everyone who wants to find out the latest developments affecting AFS and Kerberos More Info: 34 16/02/2012
  35. 35. Thank