November 2012 Tokyo OpenStack meetup was dedicated to using Gluster storage. This presentation showed the fuse mount method to integrating gluster into OpenStack. There are new drivers that have been developed that make mounting gluster volumes to instances more efficient. This presentation doesn't show how to use them.

1. OPENSTACK
STORAGE ARCHITECTURE
Dan Radez
Sr. Software Engineer, Red Hat
dradez@redhat.com
irc: radez

2. Red Hat's Community OpenStack
openstack.redhat.com

3. Storage Components
●
Database
●
Glance
●
Nova
●
Cinder
●
Swift

4. Component Architecture

5. Database :: MySQL / MariaDB
Store Options
●
Local Storage (no HA)
●
Shared Storage
●
Replicated Storage

6. Database :: MySQL / MariaDB
Local Storage
DB Node

7. Database :: MySQL / MariaDB
Shared Storage
Pacemaker
Active / Passive Failover
FIP: 10.10.10.123
DB Node*
DB Node
Storage
iSCSI or NFS
* MySQL service is running

8. Database :: MySQL / MariaDB
Shared Storage
Pacemaker
Active / Passive Failover
X
FIP: 10.10.10.123
DB Node
DB Node*
Storage
iSCSI or NFS
* MySQL service is running

9. Database :: MySQL / MariaDB
Replicated Storage
Pacemaker
Active / Passive Failover
FIP: 10.10.10.123
DB Node*
DB Node*
Master
Slave
Master / Slave Replication
* MySQL service is running

10. Database :: MySQL / MariaDB
Replicated Storage
Pacemaker
Active / Passive Failover
X
DB Node
FIP: 10.10.10.123
DB Node*
Master
Master / Slave Replication
* MySQL service is running

11. Database :: MySQL / MariaDB
Pacemaker Disclaimer
3 node minimum
for quorum

12. Glance: Image Management

13. Glance: Image Management
Store Backends
●
Local (mounted)
●
Swift
●
Cinder
●
more...
/usr/lib/python2.7/site-packages/glance/store

14. Glance: Image Management
Use GlusterFS to back Glance
# yum install -y glusterfs-fuse
# mkdir /srv/openstack
# vim /etc/fstab
gluster1:/openstack /srv/openstack glusterfs defaults,_netdev 0 0
# mount -a
# mkdir -p /srv/openstack/glance/images/
# chown -R glance:glance /srv/openstack/glance/

15. Glance: Image Management
Configure OpenStack
# vim /etc/glance/glance-api.conf
filesystem_store_datadir=/srv/openstack/glance/images/
# vim /etc/glance/glance-cache.conf
filesystem_store_datadir=/srv/openstack/glance/images/
# grep "/var/lib/glance" /etc/glance/*
glance-api.conf:#scrubber_datadir=/var/lib/glance/scrubber
glance-api.conf:#image_cache_dir=/var/lib/glance/image-cache/
glance-cache.conf:#image_cache_dir=/var/lib/glance/image-cache
glance-scrubber.conf:#scrubber_datadir=/var/lib/glance/scrubber
# service openstack-glance-api restart
# service openstack-glance-registry restart

16. Nova: Instance Management

17. Nova: Instance Management
Use GlusterFS to back Nova
# yum install -y glusterfs-fuse
# mkdir /srv/openstack
# vim /etc/fstab
gluster1:/openstack /srv/openstack glusterfs defaults,_netdev 0 0
# mount -a
# mkdir -p /srv/openstack/instances
# chown -R nova:nova /srv/openstack/instances
# vim /etc/nova/nova.conf
instances_path=/srv/openstack/instances
# service openstack-nova-compute restart

18. Cinder: Block Storage

19. Cinder: Block Storage
Drivers
●
LVM
●
NFS
●
GlusterFS
●
more...
/usr/lib/python2.7/site-packages/cinder/volume/drivers

20. Cinder: Block Storage
Example: Multiple Storage Types
Dependencies
# yum install -y glusterfs-fuse
Configuration
# vim /etc/cinder/shares.conf
gluster1:openstack
gluster2:openstack
(gluster_host.domain.com:gluster_volume_name)

21. Cinder: Block Storage
Configuration
#vim /etc/cinder/cinder.conf
enabled_backends=my_lvm,my_glusterfs
[my_lvm]
volume_group = cinder-volumes
volume_driver = cinder.volume.drivers.lvm.LVMISCSIDriver
volume_backend_name = LVM
[my_glusterfs]
volume_driver = cinder.volume.drivers.glusterfs.GlusterfsDriver
glusterfs_shares_config = /etc/cinder/shares.conf
glusterfs_sparsed_volumes = false
volume_backend_name = GLUSTER

22. Cinder: Block Storage
Restart cinder & verify
# service openstack-cinder-scheduler restart
# service openstack-cinder-volume restart
# mount | grep cinder
gluster1:openstack on /var/lib/cinder/... type fuse.glusterfs
gluster2:openstack on /var/lib/cinder/... type fuse.glusterfs

23. Cinder: Block Storage
Create cinder volume types
# cinder type-create lvm
# cinder type-key lvm set volume_backend_name=LVM
# cinder type-create glusterfs
# cinder type-key glusterfs set volume_backend_name=GLUSTER
Create cinder volume using type
# cinder create --volume-type glusterfs --display-name test 1

24. Cinder: Block Storage
Compute configuration (experimental)
# vim /etc/nova/nova.conf
debug=False
qemu_allowed_storage_drivers=gluster
# service openstack-nova-compute restart

25. Swift: Object Storage

26. Swift: Object Storage
Storage Backends
●
Swift Storage
●
GlusterFS
●
more...

27. Review

28. Resources
●
●
RDO: openstack.redhat.com
https://access.redhat.com/knowledge/docs/enUS/Red_Hat_OpenStack
●
OpenStack.org
●
TryStack.org
●
Puppetlabs.com
●
Djangoproject.com
●
http://radez.fedorapeople.org/TokyoMeetup_slides.pdf