Introduction to OpenAFSFabrizio Manfred Furuholmen                              Beolink.org
Agenda                              Beolink.org            Introduction            Architecture            Client      ...
Introduction                             Beolink.org What is a Distributed File system ? “A distributed file system takes ...
Introduction                                   Beolink.org               Andrew File System   Andrew File System is a dist...
Introduction       Beolink.org AFS is25yearsold !               5
Introduction                                                 Beolink.org                     Briefhistoryof a AFS   1983 ...
Benefits of using AFS                                                 Beolink.org   Location independence     User does n...
Benefits of using AFS                                                Beolink.org   Improved robustness to server crash   ...
Elements                                               Beolink.org  Cell  •Cell is collection of file servers and   workst...
Architecture        Beolink.org               10                       16/02/2012
Consistency                                                    Beolink.org “..That notion of callbacks gives OpenAFS a muc...
Write operation                          Beolink.org Example write operation client side  1.   create file rpc  2.   write...
Write operation                         Beolink.org Example write operation server side  1 Create file  2 Check metadata, ...
Client side            Beolink.org              Client                14
Installation                                     Beolink.org                    Supported clients  AIX 5 and 6 (though 6....
Installation                                                   Beolink.org                            ConfigurationDownlo...
Authentication                                                                         Beolink.orgAuthentication Kerberos...
Access rights                                                   Beolink.org  ACLs are only for directories ! (Files soon)...
Server side             Beolink.org              Servers                 19
Architecture        Beolink.org               20                       16/02/2012
Process                                      Beolink.org   Server Process   Function   bosserver        Basic OverSeer Ser...
Architecture                                   Beolink.orgAFS ServersTypes   Fileserver machine     file storage   Data...
Commands                    Beolink.org                VOS   PTS           Administration        FS            Commands   ...
FS                                                                                                 Beolink.orgfs: Commands...
Administration                                        Beolink.org BOS Command  Process creation  Process administration ...
Administration                      Beolink.org  VOS Command   Create volume   Volume Replication   Volume Information ...
Administration                               Beolink.orgPTS Command Create id for users and groups    Users have a posit...
Limits            Beolink.org         Limits           28
Limits                                                    Beolink.org General Limits    OpenAFS can support a maximum of...
Limits                                        Beolink.org AFS does not allow certain type of files:     Pipes     Devic...
Weaknesses                                             Beolink.org AFS is not so well suited for these situation     No r...
Introduction                              Beolink.org                      Full circle Storage is cheap. Managing storag...
Thanksto...                                               Beolink.org   Alf Watchsmann   for usage of “Introduction to AFS...
I look forwardto meeting you…                          Beolink.org                XVII European AFS meeting 2011          ...
Thank youmanfred@freemails.ch                       Beolink.org
Upcoming SlideShare
Loading in …5
×

AFS introduction

834 views
744 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
834
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Than the first question is .. With this explanation could be useful for a data center .. What do you think ?
  • Now we see how the information is archive,Volumes are similar to logical volume, the quota work as a quota and you can expand as you want, depend on the underline filesystem sizeYou can move volume wheterever you want, you can replicate volume , unfortunatly the read only copy is more a snapshoot .. Real tiem replicaYou have a specific command for handle syncronization btw volume
  • We have 2 types of services, one is name database and it is a collection of database (the name probably give you some ) and the other one is the file server also in this case .. You can understand the function. In the databae server you have 4 service, one for search and lookup the data, your information are spread around many server how can understand where is it ? Simple you use the Volume location service, this service give you the server where the information are sotred.Another service is the ptserver, it is a database for handle mapping btw id and user name and the same for groups. It also contain the group owner and member of a specific groupBu Server is the database with the information on last backup and some other related information for backup serviceThe last is deprecated, it is a special version of kerberos 4 now you can use a standard kerberos 5This is for the db server, on the other hand we have file server, witch read and save the data on the specific partition.OpenAFS is a set of file in standard file system, the block are handle with a map of inode of the partition, for this reason it is much better use separeted partitionLast component, is the client, on the client you have a kernel module and cache manager, with kerberos ticket all your request are autheticated, and handle by kernel, the cache manager controll and handle all the entry of the cche .OpenAFS works with RPC and callback that means the file server know you have a copy of a file, if the file change the fileserver break the callback to users with this mecanisim the cache is not a timer cache but a coherent.. And you have reduce the network traffic
  • We have 2 types of services, one is name database and it is a collection of database (the name probably give you some ) and the other one is the file server also in this case .. You can understand the function. In the databae server you have 4 service, one for search and lookup the data, your information are spread around many server how can understand where is it ? Simple you use the Volume location service, this service give you the server where the information are sotred.Another service is the ptserver, it is a database for handle mapping btw id and user name and the same for groups. It also contain the group owner and member of a specific groupBu Server is the database with the information on last backup and some other related information for backup serviceThe last is deprecated, it is a special version of kerberos 4 now you can use a standard kerberos 5This is for the db server, on the other hand we have file server, witch read and save the data on the specific partition.OpenAFS is a set of file in standard file system, the block are handle with a map of inode of the partition, for this reason it is much better use separeted partitionLast component, is the client, on the client you have a kernel module and cache manager, with kerberos ticket all your request are autheticated, and handle by kernel, the cache manager controll and handle all the entry of the cche .OpenAFS works with RPC and callback that means the file server know you have a copy of a file, if the file change the fileserver break the callback to users with this mecanisim the cache is not a timer cache but a coherent.. And you have reduce the network traffic
  • AFS introduction

    1. 1. Introduction to OpenAFSFabrizio Manfred Furuholmen Beolink.org
    2. 2. Agenda Beolink.org  Introduction  Architecture  Client  Administration tasks  Setup your Cell 2 16/02/2012
    3. 3. Introduction Beolink.org What is a Distributed File system ? “A distributed file system takes advantage of the interconnected nature of the network by storing files on more than one computer in the network and making them accessible to all of them..” 3 16/02/2012
    4. 4. Introduction Beolink.org Andrew File System Andrew File System is a distributed file system designed to:  handle terabytes of data  handle thousands of users  working in WAN environment 4
    5. 5. Introduction Beolink.org AFS is25yearsold ! 5
    6. 6. Introduction Beolink.org Briefhistoryof a AFS 1983 Andrew Project started at Carnegie Mellon University (CMU) 1987 Coda research work begun (based on AFS) 1988 First use of AFS version 3 (First use of AFS outside CMU) 1988 Institutional File System project at University of Michigan 1989 Transarc Corporation founded to commercialize AFS 1993 Arla project started at KungligaTekniskaHögskolan 1998 Transarc Corporation becomes wholly owned subsidiary of IBM 2000 IBM releases OpenAFS as OpenSource (IBM License) 2000 OpenAFS release version 1.0 based on Transarc 3.6 2001 OpenAFS release version 1.2 first release with better support of new operating system and fix several memory leak 2005 OpenAFS release version 1.4 with a lot of new feature 2005 AFS was discontinued from IBM 2008 U.S. Department of Energy Funds OpenAFS Development 2010 OpenAFS release version 1.6 (?) 6
    7. 7. Benefits of using AFS Beolink.org  Location independence User does not need to know which fileserver holds the file, the user only needs to know the pathname of a file.  Scalability An architectural goal of the AFS designers was client/server ratios of 114.000:1 A ratio of 2000:1 has been successfully exceeded at some sites.  Security AFS makes use of Kerberos for mutual authentication, both the service provider and the requester prove their identities AFS uses access control list (ACLs) to enable users to restrict access to their own directories, users can also create groups AFS Federation with inter cell grant  Uniform Namespace No matter where users are logged in, they see the same files  Replicates AFS Volumes Frequently accessed data can be read-only replicated on several servers (rw with osd version). Client will access the closest volume copy or load balance from a different replica 7
    8. 8. Benefits of using AFS Beolink.org  Improved robustness to server crash Clients maintain Local copies of accessed files, replicated read-only volumes on alternate fileservers can satisfy requests for a files  Wide Area Network AFS communications protocols is optimized for WAN. Retransmitting only the single bad packet in a batch of packets (RPC)  Improve system management capability Configuration changes can be made from any client in the AFS cell AFS volumes can move from one server to another without users noticing it  Operating system independent AFS client software runs on many systems (12 platforms) 8
    9. 9. Elements Beolink.org Cell •Cell is collection of file servers and workstation •The directories under /afs are cells, unique tree •Fileserver contains volumes Volumes •Volumes are "containers" or sets of related files and directories •Have size limit •3 type rw, ro, backup Mount Point Directory Server A •Access to a volume is provided through a mount point Server C •A mount point is just like a static directory Server A+B 9
    10. 10. Architecture Beolink.org 10 16/02/2012
    11. 11. Consistency Beolink.org “..That notion of callbacks gives OpenAFS a much stronger consistency guarantee than most other distributed filesystems.” Cache Manager Client-side caching lets clients access data from their local cache without going across the network for every access. Callbacks OpenAFS uses callbacks, which are a promise from the file server to the client that if the file changes, the server will contact the client to tell the client to invalidate the cached contents. 11
    12. 12. Write operation Beolink.org Example write operation client side 1. create file rpc 2. write chunks into cache (interrupted by store_data RPC) 3. read from cache 4. transfer over network 5. write to /vicepXX 12
    13. 13. Write operation Beolink.org Example write operation server side 1 Create file 2 Check metadata, permission, quota and return file path 3 write file into /vicepXX 4 Update meta data on server 5 Update db 13
    14. 14. Client side Beolink.org Client 14
    15. 15. Installation Beolink.org Supported clients AIX 5 and 6 (though 6.3)  FreeBSD 7, 8 and current  HP-UX 11.0, 11i v1 and v2 Irix 6.5  Linux 2.2, 2.4, 2.6 (ia32, ia64, x86_64, ppc, ppc64, arm, sparc, sparc64, s390, s390x) MacOS 10.3, 10.4, 10.5, 10.6 (including 64 bit). OpenBSD 4.4, 4.5, 4.6, 4.7.  Solaris 2.6, 7, 8, 9, 10, 11 (and OpenSolaris)  Also Windows ... 15
    16. 16. Installation Beolink.org ConfigurationDownload and install client package and kernel module Configure krb5 if you use it Configure AFS Files ThisCell : the name of your cell CellServDB : cell list ( of the world) cacheinfo : cache configuration (dimension and location) 16
    17. 17. Authentication Beolink.orgAuthentication Kerberos 5 kinit, retrieve a kerberos ticket aklog, convert the krb5 ticket in afs token Authentication Kaserver klog, retrieve a afs token Token operations Ticket cache: FILE:/tmp/krb5cc_0 klist, list tikets Default principal: manfred/admin@FARM.ZEROPIU.COM Valid starting Expires Service principal 08/16/10 16:03:46 08/17/10 16:03:46 krbtgt/FARM.ZEROPIU.COM@FARM.ZEROPIU.COM 08/16/10 16:03:54 08/17/10 16:03:46 afs/farm.zeropiu.com@FARM.ZEROPIU.COM  tokens, list afs token Tokens held by the Cache Manager: kdestroy, ticket destroy Users (AFS ID 15) tokens for afs@farm.zeropiu.com [Expires Aug 17 16:03] unlog, token destroyDon’t Forget  Credentials expire after some time  AFS service ticket is in the kernel memory 17
    18. 18. Access rights Beolink.org ACLs are only for directories ! (Files soon)  ACL inheritance, AFS copies ACL on a parent directory over to a new subdirectory at the time of creation  ACL awareness, not many commands are aware of ACLs (copy) ACL Permission lookup (l) List contents of directory insert (i) Add Files or directories delete (d) Delete entries in directory administer (a) Manipulate ACL for directory read (r) Read file content, query file status write (r) Write file content, change Unix permissions look (k) Full file advisory lock 18
    19. 19. Server side Beolink.org Servers 19
    20. 20. Architecture Beolink.org 20 16/02/2012
    21. 21. Process Beolink.org Server Process Function bosserver Basic OverSeer Server fileserver Serves the files volserver Serves volume data vlserver Volume location server ptserver Protection server buserver Backup server upserver Update server upclient Update client 21
    22. 22. Architecture Beolink.orgAFS ServersTypes Fileserver machine  file storage Database server machine  File and Volume localization  Groups administration  Authentication provider  Backup database Binary distribution  Master server for afs binary (specific architecture) System control machine  Time server  AFS configuration master
    23. 23. Commands Beolink.org VOS PTS Administration FS Commands BOS 23
    24. 24. FS Beolink.orgfs: Commands are:apropos search by help textcheckservers check local cells servers fs commandcheckvolumes check volumeID/name mappingscleanacl clean up access control listcopyacldiskfree copy access control list show server disk space usage  Cache management administrationexamineexportafs display file/volume status enable/disable translators to AFS  Quota managementflushflushmount flush file from cache flush mount symlink from cache ACLs managementflushvolume flush all data in volumegetcacheparms get cache usage info  Mount management on the AFS pathgetcalleraccess list callers accessgetcellstatus get cell statusgetclientaddrs get client network interface addressesgetcrypt get cache manager encryption flaggetfid get fid for file(s) setcachesize set cache sizegetserverprefs get server ranks setcbaddr configure callback connection addresshelp get help on commandslistacl list access control list setcell set cell statuslistaliases list configured cell aliases setclientaddrs set client network interface addresseslistcells list configured cells setcrypt set cache manager encryption flaglistquota list volume quota setquota set volume quotalsmount list mount point setserverprefs set server ranksmessages control Cache Manager messagesmkmount make mount point setvol set volume statusnewalias configure new cell alias storebehind store to server after file closenewcell configure new cell sysname get/set sysname (i.e. @sys) valuequota show volume quota usage uuid manage the UUID for the cache managerrmmount remove mount point whereis list files locationrxstatpeer Manage per peer RX statisticsrxstatproc Manage per process RX statistics whichcell list files cellsetacl set access control list wscell list workstations cellsetcachesize set cache size 24
    25. 25. Administration Beolink.org BOS Command  Process creation  Process administration (start, stop, status ...)  Manage Users Administrator for process  Volume check 25
    26. 26. Administration Beolink.org VOS Command  Create volume  Volume Replication  Volume Information  Move volume DON’T FORGET THERE ARE DIFFERENT PATH FOR RW and R0 RW = /afs/cell/.mount_point RO = /afs/cell/mount_point 26
    27. 27. Administration Beolink.orgPTS Command Create id for users and groups  Users have a positive number  Groups use negative number Management of Group/User membership Management metadata (group quota, flags)IMPORTANT Users can create their own groups Special groups system:anyuser system:authuser system:administrators 27
    28. 28. Limits Beolink.org Limits 28
    29. 29. Limits Beolink.org General Limits OpenAFS can support a maximum of 114.000 clients per server tmpfs no work as AFS Cache, (ramdisk work)  Max 255 partition per server (/vicepa-/vicepiv), no limits in partition size  Max 4,294,967,295 volumes per partition (this a limit of VLDB),  Max file limit per directory is 64,000 files (less than 16 characters).  Windows Limits  No integration on Microsoft DFS  No native implementation 29
    30. 30. Limits Beolink.org AFS does not allow certain type of files:  Pipes  Device files  Socket  AFS cannot do byte range locking on all platform  Client has working byte rage locks  Full file locks on the server ACLs works on directories not files (yes)  AFS does not support mandatory file locks  DES Encryption for file transport 30
    31. 31. Weaknesses Beolink.org AFS is not so well suited for these situation  No reuse of read data  Access to file larger than cache  Mostly write access  Larger numbers of directory entry changes from multiple clients 31
    32. 32. Introduction Beolink.org Full circle Storage is cheap. Managing storage is more expensive.Wide access to data is still critical.Today and into the future. 32
    33. 33. Thanksto... Beolink.org Alf Watchsmann for usage of “Introduction to AFS and its Best Practices” Please read the original presentation for a complete overview http://workshop.openafs.org/afsbpw10/ For more information read Documentation on www.openafs.org Other presentation are available on www.beolink.org 33
    34. 34. I look forwardto meeting you… Beolink.org XVII European AFS meeting 2011 HAMBURG – GERMANY Who should attend:  Everyone interested in deploying a globally accessible file system  Everyone interested in learning more about real world usage of Kerberos authentication in single realm and federated single sign-on environments  Everyone who wants to share their knowledge and experience with other members of the AFS and Kerberos communities  Everyone who wants to find out the latest developments affecting AFS and Kerberos More Info: http://www.openafs.org/ 34 16/02/2012
    35. 35. Thank youmanfred@freemails.ch Beolink.org

    ×