Downloaded 84 times
![DETAIL:
RED HAT GLUSTER STORAGE 3.1
PERF
Optimizations to enhance small file performance,
especially with small file create and write operations.
PERFSECURITYPROTOCOL
Optimizations that result in enhanced rebalance speed at large scale.
Introduction of the ability to operate with SELinux in enforcing
mode, increasing security across an entire deployment.
Support for data access via clustered, active-active NFSv4 endpoints,
based on the NFS-Ganesha project.
Enhancements to SMB 3 protocol negotiation, copy-data offload, and
support for in-flight data encryption [Sayan: what is copy-data offload?]
Small file
Rebalance
SELinux enforcing
mode
NFSv4
(multi-headed)
SMB 3 (subset of
features)
These features were introduced in the most recent release of Red Hat Gluster Storage,
and are now supported by Red Hat.
PROTOCOL](https://image.slidesharecdn.com/scalableposixfsinthecloud-20160121small-160122185259/85/Scalable-POSIX-File-Systems-in-the-Cloud-35-320.jpg)
Uploaded byRed_Hat_Storage
Scalable POSIX File Systems in the Cloud
The document discusses Red Hat's software-defined storage (SDS) portfolio, focusing on open-source systems like Ceph and Gluster for modern cloud infrastructures. It highlights the economic advantages, scalability, and flexibility of these systems while predicting significant growth in the SDS market. Key features include data protection, automated tiering, bit rot detection, and support for big data analytics, making them suitable for a variety of enterprise use cases.
More Related Content
Similar to Scalable POSIX File Systems in the Cloud
Scalable POSIX File Systems in the Cloud
- 1. SCALABLE POSIX FILESYSTEMS IN THE CLOUD Jason Callaway @jasoncallaway | blog.jasoncallaway.com January 2016
- 2. THE RED HATSTORAGE MISSION To offer a unified, open software-defined storage portfolio that delivers a range of data services for next generation workloads thereby accelerating the transition to modern IT infrastructures.
- 3. Traditional Storage Complex proprietarysilos Open, Software-Defined Storage Standardized, unified,openplatforms Custom GUI Proprietary Software Proprietary Hardware Standard Computers and Disks Standard Hardware OpenSource Software Ceph Gluster +++ Control Plane (API, GUI) ADMIN USER THE FUTURE OF STORAGE ADMIN USER ADMIN USER ADMIN USER Custom GUI Proprietary Software Proprietary Hardware Custom GUI Proprietary Software Proprietary Hardware
- 4. WHY BOTHER? PROPRIETARY HARDWARE HARDWARE-BASED INTELLIGENCE SCALE-UP ARCHITECTURE CLOSED DEVELOPMENT PROCESS Common,off-the-shelf hardware Lower cost, standardized supply chain Scale-out architecture Increased operational flexibility Software-based intelligence More programmability,agility,and control Open development process More flexible,well-integrated technology
- 5. A RISING TIDE “By2020, between 70-80% of unstructured data will be held on lower-cost storage managed by SDS environments.” “By 2019, 70% of existing storage array products will also be available as software only versions” “By 2016, server-based storage solutions will lower storage hardware costs by 50% or more.” Gartner: “IT Leaders Can Benefit From Disruptive Innovation in the Storage Industry” Innovation Insight: Separating Hype From Hope for Software-Defined Storage Innovation Insight: Separating Hype From Hope for Software-Defined Storage Market size is projected to increase approximately 20% year-over-year between 2015 and 2019. 2013 2014 2015 2016 2017 2018 2019 $1,349B $1,195B $1,029B $859B $706B $592B SDS-P MARKET SIZE BY SEGMENT $457B BlockStorage File Storage Object Storage Hyperconverged Source: IDC Software-Defined Storage is leading a shift in the global storage industry, with far-reaching effects.
- 6. Open Software-Defined Storageis a fundamental reimagining of how storage infrastructure works. It provides substantial economic and operational advantages, and it has quickly become ideally suited for a growing number of use cases. TODAY EMERGING FUTURE Cloud Infrastructure Cloud Native Apps Analytics Hyper- Convergence Containers ??? ??? THE JOURNEY
- 7. THE RED HATSTORAGE PORTFOLIO
- 8. THE RED HATSTORAGE PORTFOLIO Ceph management OPENSOURCE SOFTWARE Gluster management Ceph data services Gluster data services STANDARD HARDWARE Share-nothing, scale-out architecture provides durability and adapts to changing demands Self-managing and self-healing features reduce operational overhead Standards-based interfaces and full APIs ease integration with applications and systems Supported by the experts at Red Hat
- 9. GROWING INNOVATION COMMUNITIES lOver11M downloads in the last 12 months lIncreased developmentvelocity, authorship,and discussionhas resulted in rapid feature expansion. lContributions fromIntel, SanDisk, SUSE,and DTAG. lPresenting CephDays incities around the world and quarterly virtual Ceph Developer Summitevents. 78 AUTHORS/mo 1500 COMMITS/mo 258 POSTERS/mo 41 AUTHORS/mo 259 COMMITS/mo 166 POSTERS/mo
- 10. SanDisk sells theInfiniFlashstorage arrays, designed for use with Red Hat Ceph Storage. Optimizations contributed by SanDisk deliver high performance whichallow Ceph customers to service new workloads. Our relationship includes: lEngineering and product collaboration lCommunity thought leadership PARTNER SOLUTIONS All-flash arrays, optimized for Ceph Supermicro's Red Hat Ceph Storage optimized solutions offer durable,software- defined,scale-outstorage platforms in 1U/2U/4U form factors and are designed to maximize performance,density,and capacity Customer can expect to see: lReference architectures,validated for performance,density,and capacity lWhitepapers and datasheets that support Red Hat Storage solutions Systems designed with storage in mind Through siliconinnovationand software optimization, Intel pushes the envelope on open, software-defined storage.A key contributor,Intel recently donated significant hardware to the Ceph project. Inteldevelopment efforts have included: lSSD and performance optimizations lCephFS development Accelerating software-defined storage
- 11. Version1.3 of RedHat Ceph Storage is the firstmajor release since joining the Red Hat Storage product portfolio,and incorporates feedback from customers who have deployed inproductionat large scale. Areas of improvement: lRobustness at scale lPerformance tuning lOperational efficiency REFINEMENTS FOR PETABYTE-SCALE OPERATORS Optimized for large-scale deployments Version3.1 of Red Hat Gluster Storage contains many new features and capabilities aimed to bolster data protection, performance,security,and client compatibility. New capabilitiesinclude: lErasure coding lTiering lBit Rot detection lNVSv4 client support Enhanced for flexibility and performance
- 12. RED HAT GLUSTERSTORAGE
- 13. Nimble file storagefor petabyte-scale workloads lMachine analytics with Splunk lBig dataanalytics with Hadoop TARGET USE CASES Enterprise File Sharing lMediaStreaming lActive Archives Analytics Enterprise Virtualization OVERVIEW: RED HAT GLUSTER STORAGE Purpose-built as a scale-out file store with a straightforward architecture suitable for public, private, and hybrid cloud Simple to install and configure, with a minimal hardware footprint Offers mature NFS, SMB and HDFS interfaces for enterprise use Customer Highlight: Intuit Intuit uses Red Hat Gluster Storage to provide flexible, cost-effective storage for their industry- leading financial offerings. Rich Media & Archival
- 14. OVERVIEW: TERMINOLOGY Brick: basic unitof storage, represented by an export directory on a server in the trusted storage pool. Cluster: a group of linked computers, working together closely thus in many respects forming a single computer. FUSE: Filesystem in Userspace (FUSE) is a loadable kernel module for Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. Geo-Replication: provides a continuous, asynchronous, andincremental replication service from site to another over Local Area Networks (LANs), Wide Area Network (WANs), and across the Internet.
- 15. OVERVIEW: TERMINOLOGY Metadata: defined asdata providing information about one or more other pieces of data. There is no special metadata storage concept in GlusterFS. The metadata is stored with the file data itself. Namespace: an abstract container or environment created to hold a logical grouping of unique identifiers or symbols. Each Gluster volume exposes a single namespace as a POSIX mount point that contains every file in the cluster. Volume: a logical collection of bricks. Most of the gluster management operations happen on the volume.
- 16.
- 17. OVERVIEW: VOLUME PERMUTATIONS Distributed --Namespace is distributed horizontally across n bricks Replicated -- Namespace is synchronously replicated to an identical namespace Striped -- Namespace stripes data across bricks Dispersed -- Namespace uses Erasure Coding No Redundancy Replicated (synchronous replication) Dispersed (erasure codes) Distributed Distributed Distributed Striped Striped Distributed-Striped Distributed-Striped Geo-replicated Geo-replicated Geo-replicated
- 18. Standard replicated back-endsare very durable, and can recover very quickly, but they have an inherently large capacity overhead. Erasure coding back-ends reconstruct corrupted or lost data by using information about the data stored elsewhere in the system. Providing failure protection with erasure coding eliminates the need for RAID, consumes far less space than replication, and can be appropriate for capacity- optimized use cases. ERASURE CODING Storing more data with less hardware OBJECT/FILE 1 2 3 4 X Y ERASURE CODED POOL/VOLUME STORAGE CLUSTER Up to 75% reduction in TCO
- 19. Optimally, infrequently accesseddata is served from less expensive storage systems while frequently accessed data can be served from faster, more expensive ones. However, manually moving data between storage tiers can be time-consuming and expensive. Red Hat Gluster Storage 3.1 now supports automated promotion and demotion of data between “hot” and “cold” sub volumes based on frequency of access. TIERING Cost-effective flash acceleration OBJECT/FILE HOT SUBVOLUME (FLASH,REPLICATED) STORAGE CLUSTER COLD SUBVOLUME (ROTATIONAL, ERASURE CODED)
- 20. Bit rot detectionis a mechanism that detects data corruption resulting from silent hardware failures, leading to deterioration in performance and integrity. Red Hat Gluster Storage 3.1 provides a mechanism to scan data periodically and detect bit-rot. Using the SHA256 algorithm, checksums are computed when files are accessed and compared against previously stored values. If they do not match, an error is logged for the storage admin. BIT ROT DETECTION Detection of silent data corruption ADMIN 0 !!! 0 0 0 0 00 X
- 21. Using NFS-Ganesha, anNFS server implementation, Red Hat Gluster Storage 3.1 provides client access with simplified failover and failback in the case of a node or network failure. Supporting both NFSv3 and NFSv4 clients, NFSGanesha introduces ACLs for additional security, Kerberos authentication, and dynamic export management. SECURITY Scalable and secure NFSv4 client support CLIENT CLIENT CLIENT CLIENT NFS NFS NFS NFS NFS-GANESHA NFS-GANESHA STORAGE CLUSTER
- 22. OVERVIEW: MAXIMUMS • 64 nodesper cluster • 8 volumes per LVM RAID • XFS max recommended size for brick • 100 TB certified / 8 EB maximum on RHEL 6 • 500 TB certified / 8 EB maximum on RHEL 7 • 16 PB usable per distributed-replicated cluster • (500 TB * 64 nodes / 2 replication factor) • ~4 PB with EC2 16 TB EBS volumes • http://blog.gluster.org/category/performance/
- 23. RED HAT GLUSTERSTORAGE TARGET WORKLOADS
- 24. WHAT TOOL DOI USE? Use Case Gluster Ceph File • NFS • CIFS • FUSE Native Client Works great! Tech-preview in June OLTP-like Maybe Works great! Block Nope Works great! Object Works great! Works great! Cloud Works great! Nope Big Data Works great! Works great! Geo-replication Works great! It’s complicated
- 25. ANALYTICS Big Data analyticswith Hadoop CLOUD INFRASTRUCTURE RICH MEDIA AND ARCHIVAL SYNC AND SHARE ENTERPRISE VIRTUALIZATION Machine data analytics with Splunk Virtual machine storage with OpenStack Object storage for tenant applications Cost-effective storage for rich media streaming Active archives File sync and share with ownCloud Storage for conventional virtualization with RHEV FOCUSED SET OF USE CASES
- 26. In-place Hadoop analyticsin a POSIX compatible environment HADOOP MAP REDUCE FRAMEWORK lAllows the HortonworksData Platform 2.1 to be deployed on Red Hat Gluster Storage lHadoop toolscan operate on data in-place lAccess to the Hadoop ecosystem of tools lAccess to non-Hadoop analytics tools lConsistent operating model: Hadoop can run directly on Red Hat Gluster Storage nodes lFlexible, unified enterprise big data repository lBetter analytics (Hadoop and non-Hadoop) lFamiliar POSIX-compatible file system and tools lStart small, scale as big data needs grow lMulti-volume support (HDFS is single-volume) lUnified management (Hortonworks HDP Ambari and Red Hat Gluster Storage) FEATURES BIG DATA ANALYTICS Hadoop Distributed File System Red Hat Gluster Storage Cluster BENEFITS
- 27. High-performance, scale-out, online coldstorage for Splunk Enterprise Hot/warm data optimized for performance 10s of TB on Splunkserver DAS Cold optimized for cost, capacity and elasticity Red Hat Storage Server on commodity x86 servers lMultiple ingest options using NFS & FUSE lExpand storage pools without incurring downtime lSupport for both clustered and non-clustered configurations lRun high speed indexing and search on lSplunk’s cold data store lPay as you grow economicsfor Splunk cold data lReduce ingestion time for data with lstandard protocols l“Always online”, fast, disk-based storage pools provide constant access to historical data MACHINE DATA ANALYTICS FEATURES BENEFITS
- 28. Massively-scalable, flexible, and cost-effectivestorage for image, video, and audio content Unstructuredimage, video, and audio content lSupport for multi-petabyte storage clusters on commodityhardware lErasure coding and replication for capacity- optimized or performance-optimized pools lSupport for standard file & object protocols lSnapshots and replication capabilities for high availability and disaster recovery lProvides massive and linear scalability in on- premise or cloud environments lOffers robust data protection with an optimal blend of price & performance lStandard protocolsallow access to broadcast content anywhere, on any device lCost-effective, high performance storage for on-demand rich media content RICH MEDIA Red Hat Gluster Storage Cluster Red Hat Ceph Storage Cluster FEATURES BENEFITS
- 29. Open source, capacity-optimized archivalstorage on commodity hardware Unstructured file data lCache tiering to enable "temperature"- based storage lErasure coding to support archive and cold storage use cases lSupport for industry-standard file and object access protocols lStore data based on its access frequency lStore data on premise or in a public or hybrid cloud lAchieve durability while reducing raw capacity requirements and limiting cost lDeploy on industry-standard hardware ACTIVE ARCHIVES Red Hat Gluster Storage Cluster Red Hat Ceph Storage Cluster Unstructured object data Volume backups FEATURES BENEFITS
- 30. Powerful, software-defined, scale- out,on-premise storage for file sync and share with ownCloud Web browser lSecure file sync and share with enterprise-grade auditing and accounting lCombined solution of Red Hat Gluster Storage, ownCloud, HP ProLiant SL4550 Gen 8 servers lDeployed on-premise, managed by internal IT lAccess sync and share data from mobile devices, desktop systems, web browsers lSecure collaborationwith consumer-grade ease of use lLower risk by storing data on-premise lConform to corporate data security and compliance polices lLower totalcost of ownership with standard, high-density servers and open source FILE SYNC AND SHARE OWNCLOUD ENTERPRISE EDITION Mobile application Desktop OS RED HAT GLUSTER STORAGE FEATURES BENEFITS
- 31. The Journeyto Software-DefinedStorage INTERNAL ONLY31 Scalable, reliable storage for Red Hat Enterprise Virtualization lReliably store virtual machine images in a distributed Red Hat Gluster Storage volume lManage storage through the RHEV-M console lDeploy on standard hardware of choice lSeamlessly grow and shrink storage infrastructure when demand changes lReduce operational complexities by eliminating dependency on complex and expensive SAN infrastructures lDeploy efficiently on less expensive, easier to provision, standard hardware lAchieve centralized visibility and controlof server and storage infrastructure ENTERPRISE VIRTUALIZATION FEATURES BENEFITS
- 32.
- 33. ROADMAP: RED HAT GLUSTERSTORAGE TODAY (v3.1) V3.2 (H1-2016) FUTURE (v4.0 and beyond) Gluster 3.8, RHEL 6, 7 lDynamic provisioning of resources lInode quotas lFaster Self-heal lControlled Rebalance lSMB 3 (advanced features) lMulti-channel lAt-rest encryption lNew UI lGluster REST API lCompression lDeduplication lHighly scalable control plane lNext-gen replication/distribution lpNFS lQoS lClient side caching MGMTCOREFILESEC MGMTCOREFILEPERF Gluster 4, RHEL 7Gluster 3.7, RHEL 6, 7 lDevice Management lGeo-Replication, Snapshots lDashboard lErasure Coding lTiering lBit Rot Detection lSnap Schedule lActive/Active NFSv4 lSMB 3 (basic subset) lSELinux lSSL encryption (in-flight) MGMTCOREFILESEC
- 34. MGMT Support in theconsole for discovery, format, and creation of bricks based on recommended best practices; an improved dashboard that shows vital statistics of pools. MGMTCORECORECORE New support in the console for snapshotting and geo-replication features. New features to allow creation of a tier of fast media (SSDs, Flash) that accompanies slower media, supporting policy-based movement of data between tiers and enhancing create/read/write performance for many small file workloads. Ability to detect silent data corruption in files via signing and scrubbing, enabling long term retention and archival of data without fear of “bit rot”. Ability to schedule periodic execution of snapshots easily, without the complexity of custom automation scripts. Device management, dashboard Snapshots, Geo-replication Tiering Bit rot detection Snapshot scheduling These features were introduced in the most recent release of Red Hat Gluster Storage, and are now supported by Red Hat. CORECORE Features that enable incremental, efficient backup of volumes using standard commercial backup tools, providing time-savings over full- volume backups. Introduction of erasure coded volumes (dispersed volumes) that provide cost-effective durability and increase usable capacity when compared to standard RAID and replication. Backup hooks Erasure coding DETAIL: RED HAT GLUSTER STORAGE 3.1
- 35. DETAIL: RED HAT GLUSTERSTORAGE 3.1 PERF Optimizations to enhance small file performance, especially with small file create and write operations. PERFSECURITYPROTOCOL Optimizations that result in enhanced rebalance speed at large scale. Introduction of the ability to operate with SELinux in enforcing mode, increasing security across an entire deployment. Support for data access via clustered, active-active NFSv4 endpoints, based on the NFS-Ganesha project. Enhancements to SMB 3 protocol negotiation, copy-data offload, and support for in-flight data encryption [Sayan: what is copy-data offload?] Small file Rebalance SELinux enforcing mode NFSv4 (multi-headed) SMB 3 (subset of features) These features were introduced in the most recent release of Red Hat Gluster Storage, and are now supported by Red Hat. PROTOCOL
- 36. SCALABLE POSIX FILESYSTEM INTHE CLOUD
- 37. IN THE CLOUD GLUSTERWORKS GREAT IN AWS us-east-1a us-east-1c m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G vpc
- 38. IN THE CLOUD GLUSTERWORKS GREAT IN AWS us-east-1a us-east-1c m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G vpc 40GB Gluster Distributed Volume 40GB Gluster Distributed Volume Replication
- 41. SEEMS LEGIT… SINGLE-THREAD PERFFOR GOOGLE EARTH Type Operation Protocol Test Size MB/s Random Read NFS POSIX AIO 16MB 97.11 Random Write NFS POSIX AIO 16MB 55.12 Sequential Read NFS POSIX AIO 16MB 130.55 Sequential Write NFS POSIX AIO 16MB 56.26 Random Read FUSE Client POSIX AIO 16MB 156.34 Random Write FUSE Client POSIX AIO 16MB 83.23 Sequential Read FUSE Client POSIX AIO 16MB 146.07 Sequential Write FUSE Client POSIX AIO 16MB 82.55
- 42. DEVOPS DEPLOY & SCALEGLUSTER WITH ANSIBLE
- 43. DEVOPS SHOULD HAVE THATDONE IN AN HOUR OR SO… tinyurl.com/gluster-ansible
- 44. IN THE CLOUD GLUSTERWORKS GREAT IN AWS us-east-1a us-east-1c m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G vpc 40GB Gluster Distributed Volume 40GB Gluster Distributed Volume Replication
- 45. PUSH BUTTON SCALE OUTYOUR POSIX STORAGE! m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G m4.xlarge root 10G vpc m4.xlarge root 10G m4.xlarge root 10G
- 49.
- 50.