Openstorage with OpenStack, by Bradley

2,614 views

Published on

Session about Openstorage with OpenStack by Bradley in the 2012 OpenStack APAC Conference

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,614
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
123
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • New use cases and types of dataUnstructured dataMachine-generated files to be processed by other applicationsVideo streamingVideo surveillance
  • 2012 survey by Black Duck Software
  • Openstorage with OpenStack, by Bradley

    1. 1. OPENSTORAGE WITH OPENSTACK BRAD STONE GM, NEXENTA CHINA © Copyright Nexenta 20121
    2. 2. Enterprise Storage• Storage is 40% of an enterprise IT’s budget• Market leaders are EMC, IBM and NetApp
    3. 3. Enterprise Storage Continues to Grow
    4. 4. Storage Trends“More data will be stored in the next three years than in the previous 40,000”- Berkeley’s School of Management (2010) • Commodity components replacing specialized hardware for data storage • Storage virtualization – Commoditize by abstracting physical attributes • Global economic downturn putting pressure on capital and energy costs – Motivation to buy newer, more energy-efficient hardware
    5. 5. New use cases for data Machine-generated files Video surveillance
    6. 6. What is OpenStorage?a) Storage software is open sourceb) No vendor lock-in; components no longer need to all come from the same vendorc) Open data formatsd) All of the abovee) Some of the above
    7. 7. Top Reasons to use Open Source Software• Avoid vendor lock-in• Lower costs• Competitive advantage• Improve quality
    8. 8. Brief Biased History of OpenStorage• 1991: Linux kernel publicly announced• 2004: Google MapReduce paper• 2005: linux-iscsi and open-iscsi projects merge• 2005: OpenSolaris released• 2008: Nexenta ships first commercial product• 2008: Sun FISHworks storage appliance released• 2010: OpenStack Nova released• 2010: OpenStorage Summit features storage software vendors and commodity hardware• 2013: OpenStorage Summit in China!
    9. 9. NEXENTASTOR: A CASE STUDY FOROPENSTORAGE
    10. 10. What is NexentaStor? File and Block Access iSCSI NFS Software-based, unified storage appliance CIFS FC Leading OpenStorage solution • Runs on standard hardware Key features: • 128-bit architecture • Unlimited file size & snaps iSCSI AoE • Synchronous and asynchronous replication SAS (including replication to OpenStack Swift) FC Superior storage for virtualized environments InfiniBandNexenta is a private company with an office in Beijing!
    11. 11. Storage Pooling Discover logical disks Determine RAID requirements Pool-A Pool-B Create storage pools NFS iSCSI CIFS NFS Create datasets Assign properties (e.g. de-dup) Share (NFS, CIFS, iSCSI, FC)11
    12. 12. De-duplication • In-line de-duplication – Verify optionBlock-based • In-flight de-duplication – No verify option Flexibility De-duplication options can vary - Per volume - Per dataset - Per replication service Existing data can easily be de-duped via NexentaStor’s auto-tier capability
    13. 13. Compression• Compression done in-line at time of write• Enabled by default – Leverages typically idle processing power – Can improve overall I/O performance• Compresses only if at least 12% benefit• Many algorithms supported – LZJB for loss-less compression – GZIP 0-9 – Extensible• Compression algorithm can be changed on-the-fly – Affects future writes
    14. 14. Block support• iSCSI and FibreChannel supported – SCSI multi-pathing supported for each• Storage efficiency – Compression – Cloning – De-duplication – Thin provisioning• Support for variable block sizes
    15. 15. Technical AdvantagesDescription LimitNumber of data volumes on a system 264Maximum size of a data volume 278 bytesNumber of file systems in a data volume 264Maximum size of a file system 264 bytesNumber of devices in a data volume 264Number of files in a directory 256Maximum file size 264 bytesNumber of attributes of a file 248Maximum size of any attribute 264 bytesNumber of snapshots of a file system 26415
    16. 16. Self-healing• NexentaStor doesn’t blindly trust the data• Relies on 256-bit checksums on every block• Transparently return only the good data to the user Prevents silent data corruption• Use the good data to fix the bad copy Self-healing 0100110101 Data checked before it is returned to user X Good copy used to fix the bad copy of data 011011010116
    17. 17. Triple-parity RAID• Disk capacities doubling, but disk throughput not increasing at same rate – Result is longer time to recover from bit errors – More likely that you can be recovering from two bit errors at once – Data can then be at risk during recovery period NexentaStor provides triple-parity RAID Available with NexentaStor since 2009
    18. 18. VMDC Uses native hypervisor snapshotting functions Advantages of VMDC: 1. Provides consistent snapshots 2. Alternative is cumbersome: Take snapshot using VMware Then take snapshot on NexentaStor 3. Cloning is fast!Supports:ESX Supported Storage Protocols:Xen Server NFSHyper-V iSCSI Fibre Channel Providing visibility into your VM’s storage usage 18
    19. 19. A Broad set of Customers and PartnersCustomers Premier Partners Healthcare Finance Technology MediaEducation Internet More
    20. 20. Cloud Hosting and Telecom in AsiaCloud services do not require cloud storage 20
    21. 21. OpenStorage Adoption Finance Governments VDI Hosting, cloud compute, private cloud, virtualization R&DUniversities 2009 2010 2011 2012 2013 2014
    22. 22. NEXENTA AND OPENSTACK
    23. 23. Cloud Storage Use Cases• Storage for cloud applications• File replication of critical data for disaster recovery• Off-site backup for power and floorspace benefits in the datacenter• Geographically dispersed teams that need to write to centralized storage for long-term data access• Long-term deep data archives
    24. 24. OpenStack Nova• OpenStack – Open source cloud computing project – Started by Rackspace and NASA – Now more than 150 participating companies• OpenStack the largest open source cloud community• OpenStack Nova targets cloud computing – Includes a storage component
    25. 25. Nova Orchestrates Volume Provisioning• Provisions LUNs• Interacts with storage systems• Assigns volumes to compute nodes• Ensures LUNs are assigned to at most one VM at a time Currently only iSCSI is supported by OpenStack Nova (not FibreChannel)
    26. 26. Simplified Nova Architecture Nova Manage Nova APINova ComputeNova Schedule Rabbit MQNova Network Volume Controller NexentaStor Nova Nova Volume Database Nova driver(s) /etc/nova/nova.conf
    27. 27. Nova and NexentaStor Nova Manage NexentaStor cloning can be 1. Request to create a volume used to create a new Nova 6. Request to attach volume to a VM volume from a snapshot Nova API Virtual machine5. Volume Controller updatesdatabase with volume details Volume Controller NexentaStor Nova Nova Volume Database 3. Driver talks to NexentaStor Nova driver(s) to create volume 4. NexentaStor creates volume 2. Lookup in configuration file /etc/nova/nova.conf to determine which driver to use
    28. 28. Nexenta Nova Driver – Project Status• Implementation complete and available• Initially tested with OpenStack Diablo release• Only Netapp and Nexenta have provided Nova storage drivers• Wiki posted: – http://wiki.openstack.org/NexentaVolumeDriver• Next: Test with Cinder28 Nexenta Systems Inc. CONFIDENTIAL
    29. 29. What is Cloud Storage?• Provide storage as a service• Treat data as object or chunks• Improves ability to access unstructured data• Files divided into objects and stored across multiple systems• Defacto standard transports – Amazon S3, Google Storage, OpenStack Swift
    30. 30. Benefits of Object Storage• Scalable capacity• Relocation on demand (disaster recovery)• Access data anywhere
    31. 31. Cloud Storage• OpenStack Swift contributor• OpenStack Swift – Running Swift on NexentaStor – Technology preview downloadable from Nexenta website• Contributed Local File System patch to Swift – Leverage self-healing and advanced RAID of NexentaStor
    32. 32. NexentaStor’s Unified ArchitectureRaw Dump Swap iSCSI FC ZFS NFS CIFS ZFS Volume Emulator ZFS Posix Layer Transactional Object Layer Pooled Storage Layer Block Device Driver HDD SSD iSCSI FC
    33. 33. Unified Appliance Object Storage File BlockSwift Account ServerSwift Container Server Swift Object Server iSCSI, FibreChannel, SAS, AoE
    34. 34. Nexenta’s LFS Swift Patch• Modifies Swift ring builder logic• Allows a file system with Nexenta’s self- healing to be counted as two replicas• Can save network bandwidth when initially storing data• Can save network bandwidth by creating replacement replicas less often
    35. 35. WHERE IS OPENSTORAGEHEADING?
    36. 36. Next Generation Cloud• Nexenta is active in the cloud storage community• Defining the next generation of OpenStorage – Enabling exabyte deployments!
    37. 37. OpenStack Nova Extensions• OpenStack Nova will be extending its capability to map generic volumes to iSCSI for guest VMs to access, not just the virtualized boot volume• Interface between this new Volume service and Nexentastor Appliance should be unchanged37 Nexenta Systems Inc. CONFIDENTIAL
    38. 38. THANK YOU!!

    ×