Personal storage to enterprise storage system journey


Published on

A brief journey through storage wonderland. Notes are not visible when you view in slide share. However they would be visible when the slides are viewed in powerpoint [you have to download the slides for that]

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Storage box consists of one or more hard disks connected directly to the host or server
  • RAID provides protection against disk failures and there are many variants like RAID 1 where a simple 1:1 mirror is provided and with RAID 4, RAID 5, RAID-DP (Netapp proprietary) ,RAID 6 where 1 or 2 parity disk(s) provided for all the disks in the RAID-group. Only high-end DAS provides RAID protection against disk failure(s)SATA, SAS and FC are drive types.
  • MTBF : Mean Time between Failures
  • The graph shows that capacity with Virtualization [yellow or red] growing a lot faster that non virtualized storage [blue] In other words projection expects most of the storage expansion is to be used to deploy virtualized server and storage. It is an indication how virtualization [Server, OS] has become dominant phenomenon in enterprise data centres
  • Data is backed up regularly from the NAS servers or SAN disk arrays to tape or disk-backup appliance [Datadomain appliance or VTL appliances]. Thanks to virtualization, most of the user desktop can run multiple OS [windows and Linux] at the same time. Most of the physical server m/cs [like Dell servers] now hosts multiple virtual servers and all the server images are maintained in NAS server. This centralization helps faster deployment and better storage management. Storage is also virtualized with logical container [volume] becoming the deployment unit with multiple logical units mapped to a single physical storage volume [multiple of physical disk units with same RAID layout]. This helps better utilization of storage and easier administration of migration of data. Migration of data essentially entails moving a logical volume to different physical volume.
  • Personal storage to enterprise storage system journey

    1. 1. Personal Storage to EnterpriseStorage System - A brief journeythrough Storage Wonderland Soumen Sarkar co-founder, Idean
    2. 2. Goal and audience The presentation aims to provide a bird’s eye view on digital storage systems landscape at present It presents generic details and is not intended to be any vendor-specific If you are looking for an overview of storage-system in a data centre this presentation could be useful to you 2
    3. 3. Types of Storage Systems Directly Attached Storage Network Attached storage Storage Area Network Cloud storage - can be any combination of above three but provides RESTful interface for storage access 3
    4. 4. Directly Attached Storage (DAS) L A N S e r v e r• DAS - Storage box containing one or more harddisks connected directly to the host or server disks• DAS ranges from simple hard disk attached to Enterprise DASyour laptop to more complex enterprise-class server 4
    5. 5. Network Attached Storage (NAS) File Access is Users provided by PC NFS [for unix] or CIFS [for windows] protocol LAN Server for Data Backup Web Server Mail NAS Server DB Server Server File Server Server [unix/windows] network• All users’ request served by the application servers and storage needfrom application servers are provided by NAS box(es).• Storage Capacity planning becomes simpler and centralized 5
    6. 6. Storage Area Network (SAN) Users• SAN providesblock storage PCto theapplicationserver• Servers areconnected LANthrough SAN Web Mail DB Fileswitches (notshown in Server Server Serve Serverdiagram) r• Each Appl Diskserver has itsown Data Arrayorganization Disk/File System Array• Each applserver can planstorage need Fibre-Channel / IP SANindependently 6
    7. 7. A simplified stack comparison for DAS, NAS, SAN DAS NAS SAN Application Application Application NFS, CIFS File System Network iSCSI, FC, FCoE File System File System Network RAID RAID RAIDSAT SAT SAT A SAS FC A SAS FC A SAS FC• In DAS, applications directly call File System calls for storage needs• In NAS, applications [in different box] talk to NAS server over NFS or CIFSprotocols for all storage needs• In SAN, applications behave like DAS but File System in the server access 7storage in disk-array over iSCSI, or FC protocol
    8. 8. SATA,SAS,FC Disk Drive - Acomparative view SATA [Serial ATA] SAS [Serial FC [Fibre- Attached SCSI] channel] Performance Half-duplex, Full-duplex Full-duplex 1.5 GB/s introduced 3.0 GB /s 2.0 GB/s 3.0 GB/s in 2005 introduced, introduced 6.0 GB/s in 2007 4.0 GB in 2004 Connectivity 1 device 128 devices 127 devices 1 metre cable 6 metre cable 15 metre cable SATA only SAS & SATA FC only Redundancy Single-port HDD Dual-port HDD Dual-port HDD Availability 8x5 duty cycle 24x7 duty cycle 24x7 duty cycle 6M hrs MTBF 12M hrs MTBF 12M hrs MTBF Typical Cost Less than $1/GB $3-$4/GB $4-$5/GB [see zdnet]SATA is the cheapest, slowest and less reliable and FC iscostliest but is faster and is more reliable and is predominantly 8seen in SAN, SAS is relatively new but provides best of the
    10. 10. Personal Storage System Directly Public Cloud Attached [Google Drive, Storage Microsoft (DAS) SkyDrive etc.]• One’s personal data typically resides inside DAS and in publiccloud storage [ Gmail, Yahoo-mail, flickr, youtube, Google Drive,SkyDrive, iTune]• Data back up is taken in another standalone DAS• Size ranges from few GB to few TB at present• Further data growth will see movement of data to Cloud 10
    11. 11. Enterprise Data also is growingfast In the third quarter of 2012 (3Q12), total disk storage systems capacity shipped reach 7,104 PetaBytes or 7 Exabytes, growing 24.4% year-over-year - IDC Storage capacity growth projection: (V stands for Virtualized) source 11
    12. 12. Enterprise Storage: a schematicview DAS associated with Individual PC/Laptop Network attached Storage Area Data Centre Storage [NAS] Network [SAN] Storage Backup Infrastructure• Enterprise manages data in the order of PetaBytes and can have either NAS or SANor both• Data growth, frequency of data access are important consideration for planningstorage infrastructure• Data Replication [synchronous or asynchronous] and Data backup policy are twomajor aspects of data centre strategy 12
    13. 13. Enterprise Storage II: Multiple Data Centre helps faster disaster recovery DAS associated with DAS associated Individual PC/Laptop with Individual PC/Laptop Network Network Storage Storage attache attached Data Area Data Area d Storage Centre Network Centre Network Storage [NAS] [SAN] [SAN] [NAS] Storage Backup Storage Backup Infrastructure Infrastructure Site 1 Site 2• Multi-site Data centre helps mitigate risk of single site failure• Typically NAS of one site provides for standby NAS for the other site• Site 1 and Site 2 are connected over FC or GigEthernet• Data almost always gets synchronously replicated across sites 13
    14. 14. An emerging trendInfographics source 14
    15. 15. Enterprise Storage System III: connecting to Public Cloud DAS associated with Individual PC/Laptop DAS associated with Individual PC/Laptop Public Cloud Storage [Amazon EC3, Network Storage Microsoft Azure, Network Data attached Storage Area Rackspace] attached Storage Data Network Centre Area Storage [NAS] Centre [SAN] Network [NAS] [SAN] Storage Backup Infrastructure Storage Backup Infrastructure NAS gateway to SAN gateway to Cloud Cloud• To address rack-space limits in captive data centre enterprises often opt to move some datato public cloud• Cloud gateways like Nasuni /Twinstrata/Nirvanix/Ctera appliances connect enterprise NASor SAN seamlessly to Cloud• They usually provide a RESTful API version for storage protocols like NFS, CIFS or iSCSI 15
    16. 16. Enterprise Storage System III-b: connecting to Public Cloud Securely DAS associated with Individual DAS associated PC/Laptop with Individual Public Cloud PC/Laptop Storage [Amazon EC3, Microsoft Network Network Storage Storage Azure, Rackspace] attached Data attache Area Data Storage Centre Network Area d [NAS] Centre[SAN] Network Storage [SAN] [NAS] Storage Backup Infrastructure SAN gateway to NAS gateway Backup Infrastructure Storage to Cloud Cloud Secure gateway to Cloud• Storage infrastructure majorly deals with data in plaintext and that can be risk while dealingwith public cloud• Using VPN only makes the communication channel between cloud and data centre securebut does not protect the enterprise from intentional breach to data inside public cloud• Secure gateway that encrypts data on the fly using AES 256 cryptography addressed theneed. One may use services like Ciphercloud or NetApp Datafort appliance to secure its data 16
    17. 17. Extrapolating to Future: advent ofDCAS Many leading storage vendors are actively advocating for something called ‘private cloud’ ◦ A large part of that constitutes server and OS virtualization ◦ And storage virtualization i.e. logical provisioning of storage for the servers However analysts predict this to be a transient phase and eventually enterprises will move towards public clouds [Gartner] DCAS or Data Centre As Service will be dominant service model to be offered by the cloud vendors to enterprises ◦ Application Server, storage, security and retention of enterprise data will be provided as a single solution ◦ With enterprise users already adopting public cloud for their private data, So will only belikely scenariotheirfuture enterprise data centre it the most consistent that for enterprise data also move to public cloud appear to be a combination of captive data centre and DCAS. But Does that mean that future storage innovations will be dominated by software? 17
    18. 18. It may be when we no longer know what to do, we have cometo our real work, and that when we no longer know which wayto go, we have begun our real journey.- Wendell Berry .. And the journey continues… More storage commentary: Industry Bytes 18