• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Samba as a gateway to OpenAFS
 

Samba as a gateway to OpenAFS

on

  • 913 views

 

Statistics

Views

Total Views
913
Views on SlideShare
913
Embed Views
0

Actions

Likes
0
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Samba as a gateway to OpenAFS Samba as a gateway to OpenAFS Presentation Transcript

    • Samba as a gateway to OpenAFS Fabrizio Manfredi FuruholmenSambaXP April 2007
    • Agenda Goal Solution Gateway Architecture Gateway Configuration Integration Tools Tuning Performance Result Page 2
    • Goals Project Goal Primary goal of the project was to design and build an inexpensive storage system Requirements: Handle terabytes of data Transparent to final user Working in WAN environment Good level of scalability Page 3
    • Solution Considerations Centralize Storage (hardware solution) SAN Blockdevice interface Performance NAS Filesystem interface Shared filesystem Distributed Filesystem (software solution) Filesystem interface Single file system across multiple computer nodes Page 4
    • Solution Considerations Big Server vs Small Server (Google Techs) Small number of inexpensive fileservers provides similar performance to client side Increase in capacity are inexpensive Better manageability and redundancy. Page 5
    • Solution Storage Price Terabyte Cost (SAS/FB) Components NAS SAN DFS 14k euro NAS/SAN 4k euro DFS Storage1.5 Tb with 52.000 52.000 10 disks (110/150) Disks Size Storage 14TB 200.000 200.000 143 vs 300 SAS/FB reduce 30% 100 disks (110/150) Disks Type 3 Server Storage 9.000 250/500 SATA Disk reduce >50% 500Gb (SAS) 14 Server Storage 56.000 1Tb (SAS) Installation 4 FB interface 1.600 Software 2 Switch FB 6.000 2 Server Gw 2.000 2.000 Discount 2 Switch Gb 1.200 1.200 1.200 TOTAL for 1.5 Tb 53.200 62.800 12.200 Administration TOTAL for 14 Tb 201.200 210.800 59.200 Page 6
    • Solution Solution Distributed Filesystem AFS Free available and stable Support of large installations (>200TB with 40 milion Files) More then 20 platforms are supported Aggressive Roadmap ($350,000 per year from CSG) Samba (Gateway) AFS windows client uses internal file server emulation (slow) Clientless Fast and stable User Identity Heimdal Kerberos Autentichation (SSO) KA emulation LDAP backend 2b protocol (large kerberos ticket) Openldap Centralize storage User administration scripts (custom provisioning) Page 7
    • Solution AFS Features Transparent Access and Uniform Namespace Cell Partitions and volumes Mount Points In-use volume moves Scalability Client Caching Replication Load balance among servers while data is in use Security Authentication and secure communication Authorization and flexible access control System Management Single system interface Administration tasks without system outage Delegation Backup Page 8
    • Gateway Architeture Architeture Scalability Storage scalability (Filesystem layer) User scalability (Samba Gateway layer) Performance Load balancing Roaming user/branch office Clientless Centralized Identity Kerberos Ldap Page 9
    • Gateway Configuration Enable AFS in Samba Compile Options Enable KA server emulation --with-fake-kaserver Enable AFS ACL mapping --with-vfs-afsacl Don’t use AFS clear text password switch (old not supported) --with-afs Setting Samba Trusting (undocumented) Obtain KeyFile from AFS fileserver (/usr/afs/etc/) Import an OpenAFS KeyFile into secrets.tdb: net afs key AFSKeyFile Custom script for AFSKeyFile sync (Key rotation) Useful command (undocumented) Impersonate user, create a token for user@cell: net afs impersonate <user> <cell> Page 10
    • Gateway Configuration smb.conf Mapping Domain User<-> Pts Single domain/unique identification: afs username map = %u@zero.it Multiple domain/duplicated identification Store DOMAIN+user: afs username map = %D+%u@zero.it Store the SID in pt server: afs username map = %s@zero.it Enable AFS share afs share = yes Page 11
    • Gateway Configuration smb.conf locking Access only from samba server Samba default Access only from samba and local gw Disable oplocks , level2 oplocks .. Only with Byte-range locking on AFS client (AFS>1.5.X) Access from all system Enable strict locking option (mandatory lock) Page 12
    • Gateway Configuration Samba scalability and HA Primary server HA (DFS Root) Heartbeat VIP associated to primary Samba Server Transparent Access (MSDFS) No compile option required Enable DFS on Primary Samba server host msdfs = yes Samba Scalability DFS Proxy, Share redirection Name resolved with DNS (link is FQDN) (ex. msdfs proxy = gw1.intranet.zeropiu.itshare) DFS root , Directory link Fault tolerance (ex. ln –s msdfs:server1share1,server2share1 share1) Page 13
    • Gateway Configuration Identity Storage Heimdal integration Compile Enable ldap backend (--with-openldap) Configuration Enable ldap backend Enable 2b token for Kerberos V integration Only if have old client: enable-kaserver / afs3-salt LDAP Openldap 2.3 (SASL EXTERNAL) Extending Schema (Samba,hdb ..) nss_switch with ldap support PAM PAM Kerberos V integration Page 14
    • Integration Tools Identity Administration Custom user administration script (iauser.pl) Unix user (ldap) Samba user (ldap) Kerberos user (ldap) Pt server user Volume and mount point Groups administration script (iagroup.pl) Create unix group (ldap) Create samba group (ldap) Create pt server group Syncronization administration script (ptsSync.pl) Synchronization user from ldap to pt server Page 15
    • Performance Test Enviroment Hardware 3 FileServer Linux 2 GB of RAM, 3GHz Xeon processor 2x36Gb SAS RAID 1 for operating system partition 4x 143GB SAS RAID5 storage 2 Server Gateway Linux 2 GB of RAM, 3GHz Xeon dual processor 2x36Gb SCSI RAID 1 for operating system partition Software Samba 3.22 OpenAFS 1.4.2 IOzone 3.8 Page 16
    • Performance Samba Client write 45000 40000Client: 35000 30000 40000-45000 35000-40000 Windows XP sp2 25000 30000-35000 25000-30000 20000 20000-25000Server: 15000 15000-20000 10000 10000-15000 16384 5000 5000-10000 Linux 2.6.9 1024 0 block 0-5000 64 64 256 1024 4096 Samba 3.22 4 16384 65536 262144 kbWrite: 30-43MB/sec readRead: 40-50MB/sec 70000 60000 50000 60000-70000 40000 50000-60000 40000-50000 30000 30000-40000 20000 20000-30000 16384 10000-20000 10000 1024 0-10000 0 block 64 64 256 1024 4096 4 16384 65536 262144 kb Page 17
    • Performance write AFS Client 40000 35000Client: 30000 35000-40000 25000 Linux 2.6.9 30000-35000 20000 25000-30000 20000-25000 openafs 1.4.2 15000 15000-20000 10000 10000-15000 16384 5000-10000Server: 5000 1024 0-5000 0 block 64 64 256 Linux 2.6.9 1024 4096 4 16384 65536 262144 kb openafs 1.4.2 readWrite: 20-37MB/sec 90000 80000 70000 80000-90000 60000 70000-80000Read 50000 60000-70000 50000-60000 40000 40000-50000 Cold Cache: 22-28MB/sec 30000 30000-40000 20000 20000-30000 10000 10000-20000 Warm Cache: >45MB/sec 0 131072 16384 43 0-10000 4 16 64 256 2048 1024 4096 16384 a Page 18
    • Performance write Samba GW 25000Server: 20000 Linux 2.6.9 15000 20000-25000 15000-20000 openafs 1.4.2 10000 10000-15000 5000-10000Gateway: 5000 16384 1024 0-5000 Linux 2.6.9 0 block 64 64 256 1024 4096 4 16384 65536 262144 OpenAFS 1.4.2 kb Samba 3.22 readClient: Windows XP sp2 45000 40000 35000 40000-45000 30000 35000-40000Write: 18-25MB/sec 25000 20000 30000-35000 25000-30000 20000-25000 15000 15000-20000 10000Read 16384 10000-15000 5000 5000-10000 1024 0 0-5000Warm Cache: 30-40MB/sec 64 block 64 256 1024 4096 4 16384 65536 262144 kb Page 19
    • Performance Throughoutput Coparison write45000400003500030000 AFS25000 Samba20000 Samba-GW1500010000 read5000 90000 0 80000 2 4 8 6 2 4 8 6 72 44 88 2 4 9 9 38 76 53 51 70000 10 21 42 10 20 40 81 16 32 65 13 26 52 filesize 60000 50000 AFS Samba 40000 Samba-GW 30000 20000 10000 0 2 24 48 96 92 4 8 6 72 44 88 38 76 53 51 10 21 42 10 20 40 81 16 32 65 13 26 52 Page 20
    • Performance Tuning Samba Configuration (increase 30%) Enable socket options = TCP_NODELAY (Default) Increase SO_RCVBUF (16384) Increase SO_SNDBUF (32768) AFS Cache Manager (increase 20%) Increase block size (chunksize 19) Increase cache elements (dcache 5000) Increase server daemon (daemons 6) Increase rx packet (rxpck 2000) Increase data cache file (files 50000) Increase Cache size (cache size 4gb) Use separated disk for cache Page 21
    • Result Benefit Reduced cost Reduced storage cost 40.000 Euro (1.5TB Storage) Reduced down time Increase performance Client side Simplify System Administration task Data accessible from everywhere High security level (kerberos base) Single sign-on Disaster recovery (Volume replication) Page 22
    • Next Under Testing OpenAFS Lock subsystem, support AFS 1.5.X (Byte range) Windows client, support AFS 1.5.X Inode interface Socket communication vlserver/fileserver Memory cache Disable fsync on write (AFS 1.5.X + patch) WebDav Samba Cluster External project (www.beolink.org) Ptserver with ldap backend (ptsldap) Web Administration interface (AFS Manager) Page 23
    • The EndManfred at zeropiu.it www.beolink.org Page 24