Using CloudStack With Clustered LVM


Published on

One method KVM users have to utilize their shared block storage (SAN) is via CLVM. This document describes how.

  • Hello! Is it possible to make snapshot from the logical volume in VG on the common LUN available to some nodes?
    Are you sure you want to  Yes  No
    Your message goes here
  • There is no special integration between the cluster manager and cloudstack. If a clvm node fails, it should be fenced by the CLVM config, and cloudstack's HA features would kick in and start VMs elsewhere if they are set to HA.

    Im not entirely sure of the intent of your question, so ill also say that live migration works, but it would not save you if a node fails as there is no support for backing up live vm memory on other nodes (and the few technologies that do this are rather slow).
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi,

    Are you able to configure automatic failover on nodes backed to CLVM under KVM/cloudstack?
    Are you sure you want to  Yes  No
    Your message goes here
  • Marcus L Sorensen,

    Thanks for the slides, i am also trying to setup cloudstack with clvm, i have a question, to create clustered lvm do i have to present my iscsi lun across all the nodes? i assume so!!

    Are you sure you want to  Yes  No
    Your message goes here
  • Thanks! Watching the full talk now. :)
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Using CloudStack With Clustered LVM

  1. 1. Using CloudStack with Clustered LVMMarcus SorensenBetterServers, Inc
  2. 2. Whats LVM?● Logical Volume Manager – Built on Linux device mapper modules – Physical volumes create pools called volume groups – Volume groups provide extents from which we obtain logical volumes – Provides features like resizing, redundancy, thin provisioning, snapshots
  3. 3. What about the clustered part?● LVM can utilize a cluster manager such as corosync to syncronize LVM metadata between nodes● LVM has a configurable locking mechanism for metadata changes – /etc/lvm/lvm.conf ● locking_type – 1 is local (default) – 2 is external locking library – 3 is built-in clustered locking (depends on how LVM was compiled)● Locking handled by Corosync and DLM – CMAN in RHEL/CentOS 6.x is corosync – Ubuntu provides corosync as well● Read more about configuring a Red Hat cluster
  4. 4. Why use CLVM for Primary Storage?● Provides a way to utilize shared, block-based storage, such as iSCSI or Fibre Channel● Allows CloudStack to have a pool from which to carve out volumes● Block-based storage is high performance● Shared storage is required for live migrations● CLVM is relatively simple; it doesnt require HA resources or any of the more difficult configurations of a typical RedHat cluster.
  5. 5. Setting up the storage1. Install a base RHEL/CentOS operating system on all cluster nodes2. Install cluster software 1. RHEL/CentOS: “yum install lvm2-cluster” 2. Ubuntu: “apt-get install clvm”3. Edit /etc/lvm/lvm.conf to change locking_type4. Edit or create /etc/cluster/cluster.conf5. Start services6. Create volume group(s)7. Add into CloudStack
  6. 6. /etc/lvm/lvm.conf● Set locking_type to 3● Other tunables to be aware of: – wait_for_locks – filter
  7. 7. /etc/cluster/cluster.confExample:<?xml version="1.0"?> <cluster config_version="1" name="kvmcluster"> <cman transport="udpu"/> <clusternodes> <clusternode name="" nodeid="1"> <fence><method name="1"> <device name="fence_ipmitool" nodename="host1"/></method></fence> </clusternode> <clusternode name="" nodeid="2"> <fence><method name="1"> <device name="fence_ipmitool" nodename="host2"/></method></fence> </clusternode> <clusternode name="" nodeid="3"> <fence> <method name="1"> <device name="fence_ipmitool" nodename="host3"/></method></fence> </clusternode> </clusternodes> <fencedevices><fencedevice agent="fence_ipmitool" name="fence_ipmitool"/></fencedevices> <fence_daemon post_join_delay="60"/> <rm/></cluster>
  8. 8. Start Services● RHEL/CentOS – service cman start – service clvmd start● Ubuntu – service cman start – service clvm start● Check services: – cman_tool status – cman_tool services● Create volume groups – Just use normal LVM commands on shared block devices
  9. 9. Add into CloudStack
  10. 10. Under the hood... briefly● Volume groups and logical volumes are managed as storage pools by libvirt in LibvirtStorageAdaptor● Pools are activated/deactivated, created/destroyed dynamically as needed
  11. 11. Limitations/bugs● Snapshots consist of LVM snapshot, copy snapshot to secondary, delete snapshot● Due to nature of templating, templates unnecessarily copied to primary storage before use (fixed in master)● New dm-thin module wont be of much use with current implementations for thin- provisioned LVM volumes
  12. 12. Looking to the future...● Refactor to use Edisons storage plugin framework● Utilize dm-thin for backingstore/clone/snapshot functionality● Could provide (limited) dm-thin based snapshot functionality directly via device- mapper
  13. 13. Questions