Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using CloudStack With Clustered LVM


Published on

One method KVM users have to utilize their shared block storage (SAN) is via CLVM. This document describes how.

Using CloudStack With Clustered LVM

  1. 1. Using CloudStack with Clustered LVMMarcus SorensenBetterServers, Inc
  2. 2. Whats LVM?● Logical Volume Manager – Built on Linux device mapper modules – Physical volumes create pools called volume groups – Volume groups provide extents from which we obtain logical volumes – Provides features like resizing, redundancy, thin provisioning, snapshots
  3. 3. What about the clustered part?● LVM can utilize a cluster manager such as corosync to syncronize LVM metadata between nodes● LVM has a configurable locking mechanism for metadata changes – /etc/lvm/lvm.conf ● locking_type – 1 is local (default) – 2 is external locking library – 3 is built-in clustered locking (depends on how LVM was compiled)● Locking handled by Corosync and DLM – CMAN in RHEL/CentOS 6.x is corosync – Ubuntu provides corosync as well● Read more about configuring a Red Hat cluster
  4. 4. Why use CLVM for Primary Storage?● Provides a way to utilize shared, block-based storage, such as iSCSI or Fibre Channel● Allows CloudStack to have a pool from which to carve out volumes● Block-based storage is high performance● Shared storage is required for live migrations● CLVM is relatively simple; it doesnt require HA resources or any of the more difficult configurations of a typical RedHat cluster.
  5. 5. Setting up the storage1. Install a base RHEL/CentOS operating system on all cluster nodes2. Install cluster software 1. RHEL/CentOS: “yum install lvm2-cluster” 2. Ubuntu: “apt-get install clvm”3. Edit /etc/lvm/lvm.conf to change locking_type4. Edit or create /etc/cluster/cluster.conf5. Start services6. Create volume group(s)7. Add into CloudStack
  6. 6. /etc/lvm/lvm.conf● Set locking_type to 3● Other tunables to be aware of: – wait_for_locks – filter
  7. 7. /etc/cluster/cluster.confExample:<?xml version="1.0"?> <cluster config_version="1" name="kvmcluster"> <cman transport="udpu"/> <clusternodes> <clusternode name="" nodeid="1"> <fence><method name="1"> <device name="fence_ipmitool" nodename="host1"/></method></fence> </clusternode> <clusternode name="" nodeid="2"> <fence><method name="1"> <device name="fence_ipmitool" nodename="host2"/></method></fence> </clusternode> <clusternode name="" nodeid="3"> <fence> <method name="1"> <device name="fence_ipmitool" nodename="host3"/></method></fence> </clusternode> </clusternodes> <fencedevices><fencedevice agent="fence_ipmitool" name="fence_ipmitool"/></fencedevices> <fence_daemon post_join_delay="60"/> <rm/></cluster>
  8. 8. Start Services● RHEL/CentOS – service cman start – service clvmd start● Ubuntu – service cman start – service clvm start● Check services: – cman_tool status – cman_tool services● Create volume groups – Just use normal LVM commands on shared block devices
  9. 9. Add into CloudStack
  10. 10. Under the hood... briefly● Volume groups and logical volumes are managed as storage pools by libvirt in LibvirtStorageAdaptor● Pools are activated/deactivated, created/destroyed dynamically as needed
  11. 11. Limitations/bugs● Snapshots consist of LVM snapshot, copy snapshot to secondary, delete snapshot● Due to nature of templating, templates unnecessarily copied to primary storage before use (fixed in master)● New dm-thin module wont be of much use with current implementations for thin- provisioned LVM volumes
  12. 12. Looking to the future...● Refactor to use Edisons storage plugin framework● Utilize dm-thin for backingstore/clone/snapshot functionality● Could provide (limited) dm-thin based snapshot functionality directly via device- mapper
  13. 13. Questions