Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Samba as a gateway to OpenAFS          Fabrizio Manfredi FuruholmenSambaXP                                   April   2007
Agenda   Goal   Solution   Gateway Architecture   Gateway Configuration   Integration Tools   Tuning   Performance   Resul...
Goals                  Project Goal  Primary goal of the project was to design and build  an inexpensive storage system  R...
Solution                Considerations  Centralize Storage (hardware solution)    SAN       Blockdevice interface       Pe...
Solution               Considerations Big Server vs Small Server      (Google Techs)  Small number of inexpensive fileserv...
Solution                          Storage Price Terabyte Cost (SAS/FB)             Components            NAS       SAN    ...
Solution                                Solution   Distributed Filesystem     AFS           Free available and stable     ...
Solution                        AFS Features  Transparent Access and Uniform Namespace     Cell     Partitions and volumes...
Gateway Architeture                        Architeture Scalability    Storage scalability   (Filesystem layer)    User sca...
Gateway Configuration               Enable AFS in Samba  Compile Options    Enable KA server emulation     --with-fake-kas...
Gateway Configuration                          smb.conf    Mapping Domain User<-> Pts      Single domain/unique identifica...
Gateway Configuration                 smb.conf locking    Access only from samba server      Samba default    Access only ...
Gateway Configuration           Samba scalability and HA    Primary server HA (DFS Root)      Heartbeat      VIP associate...
Gateway Configuration                     Identity Storage    Heimdal integration       Compile           Enable ldap back...
Integration Tools             Identity Administration   Custom user administration script (iauser.pl)     Unix user (ldap)...
Performance                    Test Enviroment  Hardware    3 FileServer Linux        2 GB of RAM, 3GHz Xeon processor    ...
Performance   Samba Client                                                                                     write      ...
Performance                                                                                                  write    AFS ...
Performance                                                                                    write   Samba GW           ...
Performance                                                           Throughoutput Coparison                             ...
Performance                         Tuning  Samba Configuration (increase 30%)    Enable socket options = TCP_NODELAY (Def...
Result                        Benefit  Reduced cost    Reduced storage cost 40.000 Euro (1.5TB Storage)    Reduced down ti...
Next                      Under Testing  OpenAFS       Lock subsystem, support AFS 1.5.X (Byte range)       Windows client...
The EndManfred at zeropiu.it www.beolink.org                        Page 24
Upcoming SlideShare
Loading in …5
×

Samba as a gateway to OpenAFS

1,281 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Samba as a gateway to OpenAFS

  1. 1. Samba as a gateway to OpenAFS Fabrizio Manfredi FuruholmenSambaXP April 2007
  2. 2. Agenda Goal Solution Gateway Architecture Gateway Configuration Integration Tools Tuning Performance Result Page 2
  3. 3. Goals Project Goal Primary goal of the project was to design and build an inexpensive storage system Requirements: Handle terabytes of data Transparent to final user Working in WAN environment Good level of scalability Page 3
  4. 4. Solution Considerations Centralize Storage (hardware solution) SAN Blockdevice interface Performance NAS Filesystem interface Shared filesystem Distributed Filesystem (software solution) Filesystem interface Single file system across multiple computer nodes Page 4
  5. 5. Solution Considerations Big Server vs Small Server (Google Techs) Small number of inexpensive fileservers provides similar performance to client side Increase in capacity are inexpensive Better manageability and redundancy. Page 5
  6. 6. Solution Storage Price Terabyte Cost (SAS/FB) Components NAS SAN DFS 14k euro NAS/SAN 4k euro DFS Storage1.5 Tb with 52.000 52.000 10 disks (110/150) Disks Size Storage 14TB 200.000 200.000 143 vs 300 SAS/FB reduce 30% 100 disks (110/150) Disks Type 3 Server Storage 9.000 250/500 SATA Disk reduce >50% 500Gb (SAS) 14 Server Storage 56.000 1Tb (SAS) Installation 4 FB interface 1.600 Software 2 Switch FB 6.000 2 Server Gw 2.000 2.000 Discount 2 Switch Gb 1.200 1.200 1.200 TOTAL for 1.5 Tb 53.200 62.800 12.200 Administration TOTAL for 14 Tb 201.200 210.800 59.200 Page 6
  7. 7. Solution Solution Distributed Filesystem AFS Free available and stable Support of large installations (>200TB with 40 milion Files) More then 20 platforms are supported Aggressive Roadmap ($350,000 per year from CSG) Samba (Gateway) AFS windows client uses internal file server emulation (slow) Clientless Fast and stable User Identity Heimdal Kerberos Autentichation (SSO) KA emulation LDAP backend 2b protocol (large kerberos ticket) Openldap Centralize storage User administration scripts (custom provisioning) Page 7
  8. 8. Solution AFS Features Transparent Access and Uniform Namespace Cell Partitions and volumes Mount Points In-use volume moves Scalability Client Caching Replication Load balance among servers while data is in use Security Authentication and secure communication Authorization and flexible access control System Management Single system interface Administration tasks without system outage Delegation Backup Page 8
  9. 9. Gateway Architeture Architeture Scalability Storage scalability (Filesystem layer) User scalability (Samba Gateway layer) Performance Load balancing Roaming user/branch office Clientless Centralized Identity Kerberos Ldap Page 9
  10. 10. Gateway Configuration Enable AFS in Samba Compile Options Enable KA server emulation --with-fake-kaserver Enable AFS ACL mapping --with-vfs-afsacl Don’t use AFS clear text password switch (old not supported) --with-afs Setting Samba Trusting (undocumented) Obtain KeyFile from AFS fileserver (/usr/afs/etc/) Import an OpenAFS KeyFile into secrets.tdb: net afs key AFSKeyFile Custom script for AFSKeyFile sync (Key rotation) Useful command (undocumented) Impersonate user, create a token for user@cell: net afs impersonate <user> <cell> Page 10
  11. 11. Gateway Configuration smb.conf Mapping Domain User<-> Pts Single domain/unique identification: afs username map = %u@zero.it Multiple domain/duplicated identification Store DOMAIN+user: afs username map = %D+%u@zero.it Store the SID in pt server: afs username map = %s@zero.it Enable AFS share afs share = yes Page 11
  12. 12. Gateway Configuration smb.conf locking Access only from samba server Samba default Access only from samba and local gw Disable oplocks , level2 oplocks .. Only with Byte-range locking on AFS client (AFS>1.5.X) Access from all system Enable strict locking option (mandatory lock) Page 12
  13. 13. Gateway Configuration Samba scalability and HA Primary server HA (DFS Root) Heartbeat VIP associated to primary Samba Server Transparent Access (MSDFS) No compile option required Enable DFS on Primary Samba server host msdfs = yes Samba Scalability DFS Proxy, Share redirection Name resolved with DNS (link is FQDN) (ex. msdfs proxy = gw1.intranet.zeropiu.itshare) DFS root , Directory link Fault tolerance (ex. ln –s msdfs:server1share1,server2share1 share1) Page 13
  14. 14. Gateway Configuration Identity Storage Heimdal integration Compile Enable ldap backend (--with-openldap) Configuration Enable ldap backend Enable 2b token for Kerberos V integration Only if have old client: enable-kaserver / afs3-salt LDAP Openldap 2.3 (SASL EXTERNAL) Extending Schema (Samba,hdb ..) nss_switch with ldap support PAM PAM Kerberos V integration Page 14
  15. 15. Integration Tools Identity Administration Custom user administration script (iauser.pl) Unix user (ldap) Samba user (ldap) Kerberos user (ldap) Pt server user Volume and mount point Groups administration script (iagroup.pl) Create unix group (ldap) Create samba group (ldap) Create pt server group Syncronization administration script (ptsSync.pl) Synchronization user from ldap to pt server Page 15
  16. 16. Performance Test Enviroment Hardware 3 FileServer Linux 2 GB of RAM, 3GHz Xeon processor 2x36Gb SAS RAID 1 for operating system partition 4x 143GB SAS RAID5 storage 2 Server Gateway Linux 2 GB of RAM, 3GHz Xeon dual processor 2x36Gb SCSI RAID 1 for operating system partition Software Samba 3.22 OpenAFS 1.4.2 IOzone 3.8 Page 16
  17. 17. Performance Samba Client write 45000 40000Client: 35000 30000 40000-45000 35000-40000 Windows XP sp2 25000 30000-35000 25000-30000 20000 20000-25000Server: 15000 15000-20000 10000 10000-15000 16384 5000 5000-10000 Linux 2.6.9 1024 0 block 0-5000 64 64 256 1024 4096 Samba 3.22 4 16384 65536 262144 kbWrite: 30-43MB/sec readRead: 40-50MB/sec 70000 60000 50000 60000-70000 40000 50000-60000 40000-50000 30000 30000-40000 20000 20000-30000 16384 10000-20000 10000 1024 0-10000 0 block 64 64 256 1024 4096 4 16384 65536 262144 kb Page 17
  18. 18. Performance write AFS Client 40000 35000Client: 30000 35000-40000 25000 Linux 2.6.9 30000-35000 20000 25000-30000 20000-25000 openafs 1.4.2 15000 15000-20000 10000 10000-15000 16384 5000-10000Server: 5000 1024 0-5000 0 block 64 64 256 Linux 2.6.9 1024 4096 4 16384 65536 262144 kb openafs 1.4.2 readWrite: 20-37MB/sec 90000 80000 70000 80000-90000 60000 70000-80000Read 50000 60000-70000 50000-60000 40000 40000-50000 Cold Cache: 22-28MB/sec 30000 30000-40000 20000 20000-30000 10000 10000-20000 Warm Cache: >45MB/sec 0 131072 16384 43 0-10000 4 16 64 256 2048 1024 4096 16384 a Page 18
  19. 19. Performance write Samba GW 25000Server: 20000 Linux 2.6.9 15000 20000-25000 15000-20000 openafs 1.4.2 10000 10000-15000 5000-10000Gateway: 5000 16384 1024 0-5000 Linux 2.6.9 0 block 64 64 256 1024 4096 4 16384 65536 262144 OpenAFS 1.4.2 kb Samba 3.22 readClient: Windows XP sp2 45000 40000 35000 40000-45000 30000 35000-40000Write: 18-25MB/sec 25000 20000 30000-35000 25000-30000 20000-25000 15000 15000-20000 10000Read 16384 10000-15000 5000 5000-10000 1024 0 0-5000Warm Cache: 30-40MB/sec 64 block 64 256 1024 4096 4 16384 65536 262144 kb Page 19
  20. 20. Performance Throughoutput Coparison write45000400003500030000 AFS25000 Samba20000 Samba-GW1500010000 read5000 90000 0 80000 2 4 8 6 2 4 8 6 72 44 88 2 4 9 9 38 76 53 51 70000 10 21 42 10 20 40 81 16 32 65 13 26 52 filesize 60000 50000 AFS Samba 40000 Samba-GW 30000 20000 10000 0 2 24 48 96 92 4 8 6 72 44 88 38 76 53 51 10 21 42 10 20 40 81 16 32 65 13 26 52 Page 20
  21. 21. Performance Tuning Samba Configuration (increase 30%) Enable socket options = TCP_NODELAY (Default) Increase SO_RCVBUF (16384) Increase SO_SNDBUF (32768) AFS Cache Manager (increase 20%) Increase block size (chunksize 19) Increase cache elements (dcache 5000) Increase server daemon (daemons 6) Increase rx packet (rxpck 2000) Increase data cache file (files 50000) Increase Cache size (cache size 4gb) Use separated disk for cache Page 21
  22. 22. Result Benefit Reduced cost Reduced storage cost 40.000 Euro (1.5TB Storage) Reduced down time Increase performance Client side Simplify System Administration task Data accessible from everywhere High security level (kerberos base) Single sign-on Disaster recovery (Volume replication) Page 22
  23. 23. Next Under Testing OpenAFS Lock subsystem, support AFS 1.5.X (Byte range) Windows client, support AFS 1.5.X Inode interface Socket communication vlserver/fileserver Memory cache Disable fsync on write (AFS 1.5.X + patch) WebDav Samba Cluster External project (www.beolink.org) Ptserver with ldap backend (ptsldap) Web Administration interface (AFS Manager) Page 23
  24. 24. The EndManfred at zeropiu.it www.beolink.org Page 24

×