Cephfsglusterfs.talk
Upcoming SlideShare
Loading in...5
×
 

Cephfsglusterfs.talk

on

  • 1,093 views

Operational aspects of distributed Linux file systems like Ceph and/or GlusterFS

Operational aspects of distributed Linux file systems like Ceph and/or GlusterFS

Statistics

Views

Total Views
1,093
Views on SlideShare
1,092
Embed Views
1

Actions

Likes
1
Downloads
17
Comments
0

1 Embed 1

https://duckduckgo.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cephfsglusterfs.talk Cephfsglusterfs.talk Presentation Transcript

  • Cloud storage reloaded:Some aspects on operating distributed Linux file systems like Ceph and GlusterFS Udo Seidel
  • Agenda● Introduction/motivation● Distributed storage● Ceph and GlusterFS● Operational considerations● Summary LinuxCon Europe 05-07/11 2012 2
  • IntroductionLinuxCon Europe 05-07/11 2012 3
  • Me ;-)● Teacher of mathematics & physics● PhD in experimental physics● Started with Linux in 1996● Linux/UNIX trainer● Solution engineer in HPC and CAx environment● Head of a international team with people in Erding, Miami and Sydney @Amadeus LinuxCon Europe 05-07/11 2012 4
  • Shared File Systems● Normal business for Linux● Several implementation approaches ● Network based, e.g. NFS, CIFS ● clustered – Shared disk, e.g. OCFS2, GFS2 – Parallel/distributed, e.g. Ceph, GlusterFS LinuxCon Europe 05-07/11 2012 5
  • Distributed File Systems● Recent attention on distributed storage ● Cloud hype ● Big Data● Which solution to choose? ● Just storage or file system too?● Knock-on effects ● Technical ● Non-technical LinuxCon Europe 05-07/11 2012 6
  • Operations● Important part of the life cycle● Geek vs. Enterprise● Technical challenges and more LinuxCon Europe 05-07/11 2012 7
  • Distributed Storage LinuxCon Europe 05-07/11 2012 8
  • History● Reviewing storage task responsibilities ● Block allocation ● Space management● Extension of SCSI standard ● Object based storage ● Meta-Data handling separated from data management LinuxCon Europe 05-07/11 2012 9
  • Object based storage● Storage objects quite general ● Partition, file, ... ● Unique identifier● OSD (Object based Storage Device) ● Hardware -> original trigger ● Software -> common implementation● Interfaces to existing technology ● Part of the distributed storage solution ● Additional development of existing approaches LinuxCon Europe 05-07/11 2012 10
  • Distributed storage: Paradigm changes● Block -> Object● Central -> Distributed ● Few -> Many ● Big -> Small LinuxCon Europe 05-07/11 2012 11
  • Distributed storage – Now what?!?● Several implementations● Different functions● Support models● Storage vendors initiatives● Relation to Linux distributions LinuxCon Europe 05-07/11 2012 12
  • Ceph and GlusterFS LinuxCon Europe 05-07/11 2012 13
  • Ceph and GlusterFS● Sufficient community presence● Picked up by Enterprise Linux vendors● Good amount of commonalities but still quite different LinuxCon Europe 05-07/11 2012 14
  • Ceph and GlusterFS – High level● Server part ● Block storage -> storage server (OSD) ● (POSIX) file system -> Meta data ● HA -> replication and distribution● Client part ● (POSIX) file system ● Storage layer LinuxCon Europe 05-07/11 2012 15
  • CephLinuxCon Europe 05-07/11 2012 16
  • GlusterFSLinuxCon Europe 05-07/11 2012 17
  • Ceph and GlusterFS – some details CEPH GLUSTERFS Server implementation userspace userspace Client implementation userspace/FUSE and userspace/FUSE native kernel File system access CEPH protocol suite GlusterFS and NFS protocol Block device access yes RFP ongoing QEMU integration yes Under development Replication traffic OSD network Client-server network Interconnect IP based IP based and/or Infiniband Encryption no not really Back-end file system XFS ..... BTRFS ... ext4 XFS ........ext4 Storage object CRUSH object file (except for striping) GEO replication not really yes LinuxCon Europe 05-07/11 2012 18
  • Ceph and GlusterFS – more details CEPH GLUSTERFS Online growth/shrink yes yes POSIX yes alike Concurrent workload technically yes technically yes NAS replacement technically yes technically yes SAN replacment technically yes no Hadoop integration yes yes Openstack integration yes yes Community version yes yes available Production readiness Block layer File system layer Service provider Vendor and other Vendor and other ... ... ... LinuxCon Europe 05-07/11 2012 19
  • Operational considerations LinuxCon Europe 05-07/11 2012 20
  • Status quo “pre distributed storage”● Server and Storage ● Clear responsibilities ● Known area: SCSI, NFS, FC ● Strict border lines – Technically – organizationally● Storage hardware and topology ● Less focus on local drives ● Dedicated/separated storage network LinuxCon Europe 05-07/11 2012 21
  • Changes with distributed storage● New paradigm● Big overlap between server and storage ● Technical implication ● Organizational structure● Strong network dependency ● Topology ● physics● Support and supported setups LinuxCon Europe 05-07/11 2012 22
  • Challenge one: server vs. storage● Need for ● Server admin with strong storage skills ● Storage admin with strong O/S skills● Review of ● Current organizational setup ● Strategy for traditional storage ● Operation procedures ● HA architecture LinuxCon Europe 05-07/11 2012 23
  • Challenge one: possible answers● DevOps model for O/S and storage admins ● Best of both worlds ● Better common understanding ● Potential of process simplification ● BUT organizational changes needed! LinuxCon Europe 05-07/11 2012 24
  • Challenge one: possible answers● Appliance model for O/S and storage admins ● Less internal changes ● Potential of reusing existing processes ● BUT less control and visibility LinuxCon Europe 05-07/11 2012 25
  • Challenge two: infrastructure● Dedicated vs. Shared● Responsibilities ● server vs. storage vs. network● Transport medium● Local setup ● Spinning disks vs. SSD ● RAID? ● HA configuration of single server LinuxCon Europe 05-07/11 2012 26
  • Challenge two: possible answers● Different flavour of performance vs. cost topic ● Dedicated vs. Shared ● Transport medium ● Spinning disks vs. SSD ● HA on server level LinuxCon Europe 05-07/11 2012 27
  • Challenge two: possible answers● Team up with network organization● Single server full blown SPOF on purpose● Think big LinuxCon Europe 05-07/11 2012 28
  • Challenge three: support● Service provider● Application support LinuxCon Europe 05-07/11 2012 29
  • Challenge three: possible answers● Combine with Linux vendor support ● Single point of contact ● Leverage vendor QA● Hide the new technology ● XaaS ● Appliance approach ● File system level● Reminder: Think big LinuxCon Europe 05-07/11 2012 30
  • Summary ... kind of ... LinuxCon Europe 05-07/11 2012 31
  • The message● New world with several challenges ● Dont underestimate them ● Dont overestimate them● Technical revolution ● requires non-technical changes ● Dont underestimate the potential● Think big! ;-)● Get started! LinuxCon Europe 05-07/11 2012 32
  • References● http://www.ceph.com● http://www.gluster.org● “your business/company” LinuxCon Europe 05-07/11 2012 33
  • Thank you!LinuxCon Europe 05-07/11 2012 34