Openstorage with OpenStack, by Bradley
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Openstorage with OpenStack, by Bradley



Session about Openstorage with OpenStack by Bradley in the 2012 OpenStack APAC Conference

Session about Openstorage with OpenStack by Bradley in the 2012 OpenStack APAC Conference



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • New use cases and types of dataUnstructured dataMachine-generated files to be processed by other applicationsVideo streamingVideo surveillance
  • 2012 survey by Black Duck Software

Openstorage with OpenStack, by Bradley Presentation Transcript

  • 2. Enterprise Storage• Storage is 40% of an enterprise IT’s budget• Market leaders are EMC, IBM and NetApp
  • 3. Enterprise Storage Continues to Grow
  • 4. Storage Trends“More data will be stored in the next three years than in the previous 40,000”- Berkeley’s School of Management (2010) • Commodity components replacing specialized hardware for data storage • Storage virtualization – Commoditize by abstracting physical attributes • Global economic downturn putting pressure on capital and energy costs – Motivation to buy newer, more energy-efficient hardware
  • 5. New use cases for data Machine-generated files Video surveillance
  • 6. What is OpenStorage?a) Storage software is open sourceb) No vendor lock-in; components no longer need to all come from the same vendorc) Open data formatsd) All of the abovee) Some of the above
  • 7. Top Reasons to use Open Source Software• Avoid vendor lock-in• Lower costs• Competitive advantage• Improve quality
  • 8. Brief Biased History of OpenStorage• 1991: Linux kernel publicly announced• 2004: Google MapReduce paper• 2005: linux-iscsi and open-iscsi projects merge• 2005: OpenSolaris released• 2008: Nexenta ships first commercial product• 2008: Sun FISHworks storage appliance released• 2010: OpenStack Nova released• 2010: OpenStorage Summit features storage software vendors and commodity hardware• 2013: OpenStorage Summit in China!
  • 10. What is NexentaStor? File and Block Access iSCSI NFS Software-based, unified storage appliance CIFS FC Leading OpenStorage solution • Runs on standard hardware Key features: • 128-bit architecture • Unlimited file size & snaps iSCSI AoE • Synchronous and asynchronous replication SAS (including replication to OpenStack Swift) FC Superior storage for virtualized environments InfiniBandNexenta is a private company with an office in Beijing!
  • 11. Storage Pooling Discover logical disks Determine RAID requirements Pool-A Pool-B Create storage pools NFS iSCSI CIFS NFS Create datasets Assign properties (e.g. de-dup) Share (NFS, CIFS, iSCSI, FC)11
  • 12. De-duplication • In-line de-duplication – Verify optionBlock-based • In-flight de-duplication – No verify option Flexibility De-duplication options can vary - Per volume - Per dataset - Per replication service Existing data can easily be de-duped via NexentaStor’s auto-tier capability
  • 13. Compression• Compression done in-line at time of write• Enabled by default – Leverages typically idle processing power – Can improve overall I/O performance• Compresses only if at least 12% benefit• Many algorithms supported – LZJB for loss-less compression – GZIP 0-9 – Extensible• Compression algorithm can be changed on-the-fly – Affects future writes
  • 14. Block support• iSCSI and FibreChannel supported – SCSI multi-pathing supported for each• Storage efficiency – Compression – Cloning – De-duplication – Thin provisioning• Support for variable block sizes
  • 15. Technical AdvantagesDescription LimitNumber of data volumes on a system 264Maximum size of a data volume 278 bytesNumber of file systems in a data volume 264Maximum size of a file system 264 bytesNumber of devices in a data volume 264Number of files in a directory 256Maximum file size 264 bytesNumber of attributes of a file 248Maximum size of any attribute 264 bytesNumber of snapshots of a file system 26415
  • 16. Self-healing• NexentaStor doesn’t blindly trust the data• Relies on 256-bit checksums on every block• Transparently return only the good data to the user Prevents silent data corruption• Use the good data to fix the bad copy Self-healing 0100110101 Data checked before it is returned to user X Good copy used to fix the bad copy of data 011011010116
  • 17. Triple-parity RAID• Disk capacities doubling, but disk throughput not increasing at same rate – Result is longer time to recover from bit errors – More likely that you can be recovering from two bit errors at once – Data can then be at risk during recovery period NexentaStor provides triple-parity RAID Available with NexentaStor since 2009
  • 18. VMDC Uses native hypervisor snapshotting functions Advantages of VMDC: 1. Provides consistent snapshots 2. Alternative is cumbersome: Take snapshot using VMware Then take snapshot on NexentaStor 3. Cloning is fast!Supports:ESX Supported Storage Protocols:Xen Server NFSHyper-V iSCSI Fibre Channel Providing visibility into your VM’s storage usage 18
  • 19. A Broad set of Customers and PartnersCustomers Premier Partners Healthcare Finance Technology MediaEducation Internet More
  • 20. Cloud Hosting and Telecom in AsiaCloud services do not require cloud storage 20
  • 21. OpenStorage Adoption Finance Governments VDI Hosting, cloud compute, private cloud, virtualization R&DUniversities 2009 2010 2011 2012 2013 2014
  • 23. Cloud Storage Use Cases• Storage for cloud applications• File replication of critical data for disaster recovery• Off-site backup for power and floorspace benefits in the datacenter• Geographically dispersed teams that need to write to centralized storage for long-term data access• Long-term deep data archives
  • 24. OpenStack Nova• OpenStack – Open source cloud computing project – Started by Rackspace and NASA – Now more than 150 participating companies• OpenStack the largest open source cloud community• OpenStack Nova targets cloud computing – Includes a storage component
  • 25. Nova Orchestrates Volume Provisioning• Provisions LUNs• Interacts with storage systems• Assigns volumes to compute nodes• Ensures LUNs are assigned to at most one VM at a time Currently only iSCSI is supported by OpenStack Nova (not FibreChannel)
  • 26. Simplified Nova Architecture Nova Manage Nova APINova ComputeNova Schedule Rabbit MQNova Network Volume Controller NexentaStor Nova Nova Volume Database Nova driver(s) /etc/nova/nova.conf
  • 27. Nova and NexentaStor Nova Manage NexentaStor cloning can be 1. Request to create a volume used to create a new Nova 6. Request to attach volume to a VM volume from a snapshot Nova API Virtual machine5. Volume Controller updatesdatabase with volume details Volume Controller NexentaStor Nova Nova Volume Database 3. Driver talks to NexentaStor Nova driver(s) to create volume 4. NexentaStor creates volume 2. Lookup in configuration file /etc/nova/nova.conf to determine which driver to use
  • 28. Nexenta Nova Driver – Project Status• Implementation complete and available• Initially tested with OpenStack Diablo release• Only Netapp and Nexenta have provided Nova storage drivers• Wiki posted: –• Next: Test with Cinder28 Nexenta Systems Inc. CONFIDENTIAL
  • 29. What is Cloud Storage?• Provide storage as a service• Treat data as object or chunks• Improves ability to access unstructured data• Files divided into objects and stored across multiple systems• Defacto standard transports – Amazon S3, Google Storage, OpenStack Swift
  • 30. Benefits of Object Storage• Scalable capacity• Relocation on demand (disaster recovery)• Access data anywhere
  • 31. Cloud Storage• OpenStack Swift contributor• OpenStack Swift – Running Swift on NexentaStor – Technology preview downloadable from Nexenta website• Contributed Local File System patch to Swift – Leverage self-healing and advanced RAID of NexentaStor
  • 32. NexentaStor’s Unified ArchitectureRaw Dump Swap iSCSI FC ZFS NFS CIFS ZFS Volume Emulator ZFS Posix Layer Transactional Object Layer Pooled Storage Layer Block Device Driver HDD SSD iSCSI FC
  • 33. Unified Appliance Object Storage File BlockSwift Account ServerSwift Container Server Swift Object Server iSCSI, FibreChannel, SAS, AoE
  • 34. Nexenta’s LFS Swift Patch• Modifies Swift ring builder logic• Allows a file system with Nexenta’s self- healing to be counted as two replicas• Can save network bandwidth when initially storing data• Can save network bandwidth by creating replacement replicas less often
  • 36. Next Generation Cloud• Nexenta is active in the cloud storage community• Defining the next generation of OpenStorage – Enabling exabyte deployments!
  • 37. OpenStack Nova Extensions• OpenStack Nova will be extending its capability to map generic volumes to iSCSI for guest VMs to access, not just the virtualized boot volume• Interface between this new Volume service and Nexentastor Appliance should be unchanged37 Nexenta Systems Inc. CONFIDENTIAL
  • 38. THANK YOU!!