LinuxTag 2013 5Me ;-)● Teacher of mathematics & physics● PhD in experimental physics● Started with Linux in 1996● Linux/UNIX trainer● Solution engineer in HPC and CAx environment● Head of the Linux Strategy team @Amadeus
LinuxTag 2013 6Storage: History● Reviewing storage task responsibilities● Block allocation● Space management● Extension of SCSI standard● Object based storage● Meta-Data handling separated from datamanagement
LinuxTag 2013 7Object based storage● Storage objects quite general● Partition, file, ...● Unique identifier● OSD (Object based Storage Device)● Hardware -> original trigger● Software -> common implementation● Main component of distributed file systems
LinuxTag 2013 8Distributed storage:Paradigm changes● Block -> Object● Central -> Distributed● Few -> Many● Big -> Small● Server <-> Storage
LinuxTag 2013 9Distributed File Systems● Recent attention on distributed storage● Cloud hype● Big Data● See also CEPH talks
LinuxTag 2013 10Distributed storage – Now what?!?● Several implementations● Different functions● Support models● Storage vendors initiatives● Relation to Linux distributionsHere and now ==> GlusterFS
LinuxTag 2013 17The Brick● Trust each other● Interconnect● TCP/IP and/or RDMA/Infiniband● Dedicated file systems on GlusterFS server● XFS recommended, EXT4 works too● Extended attributes a must● Two main processes/daemons● glusterd and glusterfsd
LinuxTag 2013 18The Translator● One per purpose● Replication● POSIX● Quota● I/O behaviour● Chained -> brick graph● Technically: configuration
LinuxTag 2013 19The Volume● Service unit● Layer of configuration● distributed, replicated, striped, ...● NFS● Cache● Permissions● ....
LinuxTag 2013 24Meta Data● 2 kinds● More of local file system style● Related to distributed nature● Some stored in backend file system● Permissions● Time stamps● Distribution/replication● Some calculated on the fly● Brick location
LinuxTag 2013 25Elastic Hash Algorithm● Based on file names● Name space divided● Full brick handled via relinking● Stored in extended attributes● Client needs to know topology
LinuxTag 2013 27Self-Healing● On demand vs. Scheduled● File based● Based on extended attributes● Split-brain● Quorum function● Sometimes: manual intervention
LinuxTag 2013 28Geo replication● Asynchronous● Based on rsync/ssh● Master-Slave● If needed: cascading● One way street● Clocks in sync!
LinuxTag 2013 29From files to objects● Introduced with version 3.3● Hard links with some hierarchy● Re-uses GFID (inode number)● UFO● Unified File and Object● Combination with RESTful API● S3 and swift compatible
LinuxTag 2013 30Operations:Growth, shrinkage .. failures● A Must!● Easy● Rebalance!● Order of servers important