This document discusses integrating the open source GlusterFS distributed file system with QEMU virtualization and the oVirt virtualization management platform. It provides an overview of GlusterFS and how it can be used as a virtual machine image store. It describes integrating GlusterFS with QEMU using different methods and shows performance results. It then outlines the architecture and features for managing GlusterFS volumes from within oVirt, including a REST API and upcoming integration plans. The goal is to provide an open alternative for large scale centralized management of server and desktop virtualization.

1. Integrating GlusterFS, qemu and
oVirt
Vijay Bellur
Gluster Workshop – LinuxCon EU 2013

2. The Future of Storage is Open for Business 2
Agenda
➢Introduction
➢GlusterFS as VM Image Store
➢Integration with QEMU
➢Integration with oVirt
➢Q&A

3. The Future of Storage is Open for Business 3
GlusterFS
Global namespace
Scale-out clustered
storage building
blocks
Support thousands
of clients
Access using
GlusterFS and NFS
protocols
Linear performance
scaling

4. The Future of Storage is Open for Business 4
KVM and Qemu
KVM is a full
virtualization solution
for Linux on x86
hardware containing
virtualization
extensions.
Qemu is the driver for
KVM virtualization

5. The Future of Storage is Open for Business 5
➢ Large scale, centralized
management for server
and desktop
virtualization
➢ Open source alternative
to vCenter / vSphere
➢ Focus on KVM
oVirt

6. The Future of Storage is Open for Business 6
GlusterFS 3.3
➢ Granular data self heal
➢ Pro-active self healing
➢ Online data migration with remove-brick
➢ Support for O_DIRECT in FUSE

7. The Future of Storage is Open for Business 7
GlusterFS 3.4
➢Improved write-behind behavior
➢libgfapi with support for DVM
➢Block Device Translator
➢Support for volume/cluster management from oVirt
➢FUSE
➢ Scatter – Gather patchset
➢ gid caching
➢ Configurable fuse queue length
➢ aio support in posix translator

8. The Future of Storage is Open for Business 8
GlusterFS 3.5
➢ fallocate() support
➢ zerofill support in libgfapi
➢ Improved block device translator
➢ File snapshots

9. The Future of Storage is Open for Business 9
QEMU & GlusterFS Integration
Integration possible through:
➢ GlusterFS native client - possible with qemu 1.2 and
GlusterFS 3.3
➢ GlusterFS NFS – possible with qemu 1.2 and GlusterFS
3.3
➢ libgfapi – possible with qemu 1.3 and GlusterFS 3.4

10. The Future of Storage is Open for Business 10
QEMU & GlusterFS Integration
➢ Source: LPC 2012

11. The Future of Storage is Open for Business 11
QEMU & GlusterFS Integration
➢FIO Numbers (Seq read, 4 files with direct io, qemu
options: if=virtio, cache=none)
Aggregate
BW(kB/s)
Min BW(kB/s) Max BW(kB/s)
Base 63076 15769 17488
FUSE mount 29392 7348 9266
QEMU-
GlusterFS
native
integration
53609 13402 14909

12. The Future of Storage is Open for Business 12
Architecture
BrowserBrowser
EngineEngine
Directory
Services
Directory
Services
Storage
Server 1
Storage
Server 1
CLICLI
Python
SDK
Python
SDK
R
E
S
T
R
E
S
T
GWT ServletGWT Servlet
api
gwtrpc
over https
REST
over https
xmlrpc
V
D
S
M
V
D
S
M
Storage
Server 2
Storage
Server 2
V
D
S
M
V
D
S
M
Storage
Server n
Storage
Server n
V
D
S
M
V
D
S
M
oVirt & GlusterFS - Architecture

13. The Future of Storage is Open for Business 13
oVirt & GlusterFS
●
New feature in oVirt 3.1
http://wiki.ovirt.org/wiki/Features/Gluster_Support
●
Cluster Configuration Modes
●
Virtualization only
●
Gluster only
●
Virtualization + Gluster (default)
●
New entities (Volumes, Bricks, Volume Options)
●
VDSM verbs for gluster management

14. The Future of Storage is Open for Business 14
Architecture
BrowserBrowser
EngineEngine
Directory
Services
Directory
Services
Storage
Server 1
Storage
Server 1
CLICLI
Python
SDK
Python
SDK
R
E
S
T
R
E
S
T
GWT ServletGWT Servlet
api
gwtrpc
over https
REST
over https
xmlrpc
V
D
S
M
V
D
S
M
Storage
Server 2
Storage
Server 2
V
D
S
M
V
D
S
M
Storage
Server n
Storage
Server n
V
D
S
M
V
D
S
M
oVirt & GlusterFS - Architecture

15. The Future of Storage is Open for Business 15
oVirt Gluster Management Features
➢Cluster Management
● Create Cluster
● Add / Remove Storage Servers
● Delete Cluster
➢Volume Management
● Create Volume
● Add / Remove bricks
● Start / Stop / Delete volume

16. The Future of Storage is Open for Business 16
oVirt Gluster Management Features
➢Advanced Search
➢Multi Level Administration & RBAC
➢Audit & Alerts
➢ReST API
➢Python SDK & CLI

17. The Future of Storage is Open for Business 17
ReST API

18. The Future of Storage is Open for Business 18
Events

19. The Future of Storage is Open for Business 19
➢
Create PosixFS storage domain (VFS type = glusterfs)
Optimize volume for virt store (Sets optimization options on volume)
Gluster as Virt Store

20. The Future of Storage is Open for Business 20
Upcoming features (3.2 and beyond)
● Async Tasks
● UFO configuration
Upcoming features (oVirt 3.4 and beyond)

21. The Future of Storage is Open for Business 21
Upcoming features (3.2 and beyond)
● Storage Device management
● Reports
● Gluster native storage domain
● Volume Quotas
Upcoming features (oVirt 3.2 and beyond)

22. 22
Contribute
● Git repository
git://gerrit.ovirt.org/ovirt-engine
git://gerrit.ovirt.org/vdsm
git://git.gluster.com/glusterfs.git
● IRC Channels
● #ovirt on oftc
● #gluster on freenode

23. The Future of Storage is Open for Business 23
Thank You!
Q & A