The document discusses GPU-accelerated Virtual Desktop Infrastructure (VDI) on OpenStack by Nimbis Services, Inc., outlining challenges and solutions for utilizing GPUs in virtual environments. It details various technologies and tests conducted, including integrations with Microsoft Hyper-V, XenServer, and open-source solutions, while highlighting the current state of development and the need for community support. The summary concludes with a statement on the feasibility of GPU-accelerated VDI pools within OpenStack, although production readiness remains an issue.

1. Page 1
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
GPU-Accelerated VDI on OpenStack
OpenStack DC MeetUp
June 18, 2013
Nimbis Services, Inc.
http://www.nimbisservices.com
Brian Schott, CTO
Brian.Schott@NimbisServices.com

2. Page 2
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
About Nimbis Services
Nimbis Services is lowering the barriers to high-
performance computing (HPC) and offering a secure
marketplace for technical computing in the cloud.
Nimbis partners with leading HPC & cloud providers, software
vendors, and domain experts to configure and broker secure
on-demand pay-as-you-go access to applications.

3. Page 3
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Challenges for GPU-Accelerated Virtual Desktop Infrastructure (VDI)
on OpenStack
 Making the GPU available to a virtual machine.
(3 basic approaches, ok, maybe 4).
 Bare Metal / Containers (lxc)
 PCI Pass-Through
 Para-virtualization (proxy device drivers)
 Remote Display Protocols (X, NX, VirtualGL, WebGL)
 Remote desktop protocol that supports GPU acceleration.
 Host and Guest Combinations: Linux or Windows
 Remote Clients: Linux, Windows, OSX, Web Browser, Tablet
 Networking: “The Joys of Firewalls”, or “Any Port as Long as it is 443”
 Licenses: “This is too good to be free.”
 Provisioning GPU resources from within OpenStack.
 Nova compute virtualization driver (plugin).
 GPU instance types, resource discovery, and scheduling.
 This is still under development in the community.
 Full disclosure: Nimbis hand-deployed the GPU instances for testing.

4. Page 4
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Nimbis Testbed Hardware
 “Production” Cisco System
 (1) UCS C220 M3 – dual quad-core 3.3GHz
Intel Xeon E5-2643, 192GB SDRAM, 2TB
SAS-7K, dual channel 10GbE interface.
 (2) UCS C220 M3 – dual quad-core 2.4GHz
Intel Xeon E5-2609, 192GB SDRAM, 2TB
SAS-7K, dual channel 10GbE interface.
 (3) UCS C240 M3 – dual quad-core 2.4GHz
Intel Xeon E5-2609, 192GB SDRAM, 6TB
SAS-7K, dual-channel 10GbE.
• (1) Nvidia GRID K2 Board
 (2) Nexus 3064 10GbE Switches
 (2) ASA 5520 Adaptive Security Appliances
 (2) Cisco Catalyst 2960S 1GbE switches
 “Development” Cisco System
 (3) Cisco UCS C200 M2
 (1) Nexus 3064 10GbE Switch
 (1) ASA 5520 Adaptive Security Appliance
 (1) Cisco Catalyst 2960S 1GbE switch
Cores: 72
Memory: 1440 GB
Disk: 36 TB
Servers: 9*
Switches: 9*
* What do you
expect with a
Cisco testbed?

5. Page 5
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Nvidia GRID GPU Virtualization (Kepler Architecture)
Source: http://www.nvidia.com/object/grid-boards.html
Source: 1http://www.nvidia.com/content/PDF/kepler/NV_DS_Tesla_KCompute_Arch_May_2012_LR.pdf
GPU Virtualization¹
GRID boards feature
the NVIDIA Kepler
architecture that, for the
first time, allows
hardware virtualization
of the GPU. This
means multiple users
can share a single GPU,
improving user density
while providing true
PC performance and
compatibility.

6. Page 6
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
GPU-Accelerated VDI Approaches Considered
 Customer Requirements
 Combination of Windows and Linux guest images.
 Applications using OpenGL on both Windows and Linux.
 Variety of Windows (95, 7, 8) clients, OSX clients, and web browsers
(IE7, IE8, Firefox, Chrome, Safari).
 Variety of firewall and IT/FSO software challenges.
 Technologies Tested
 Microsoft Hyper-V and RemoteFX
 XenServer with VNC / RDP / Guacamole
 Open Source Xen (libvirt) on Ubuntu Linux with VNC
 Possible Alternatives
 Linux Containers (lxc)
• USC/ISI team has the blueprint and code targeted for Havana.
• No Windows support.
 XenDesktop with HDX 3D Pro (not tested)
• Requires a custom client, Citrix Receiver, OpenGL support unclear.
• Does not integrate with OpenStack.

7. Page 7
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Microsoft Hyper-V and RemoteFX
 Technology
 Windows Host GPU device with para-virtualized driver for windows guests.
 Host / Hypervisor
 Microsoft Windows Server 2012 Datacenter Edition (RemoteFX license).
 Latest Nvidia driver for Windows.
 Guests
 No GPU pass-through available or driver for Linux guests.
 Accelerates Windows DirectX calls, does software OpenGL rendering.
 Clients
 Good performance with standard Windows RDC > 8.0.
 Good performance from FreeRDP* OSX client, not Microsoft client.
 iOS and Android? Not tested, but FreeRDP clients exist on both.
 OpenStack Status
 Integrates with OpenStack Hyper-V nova compute drivers.
 Nova scheduler support for Hyper-V extra specs not on blueprint radar.
 Possible to use today with custom nova scheduler plugin.
* http://www.freerdp.com

8. Page 8
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Hyper-V and RemoteFX Benchmarks with Nvidia GRID K2
GFXBench Test Metric
RDP Without
RemoteFX
NVidia
Quadro 600
(~$150)*
RDP With
RemoteFX
NVidia
Quadro K5000
(~$1800)*
Spark Lines
Fill rate - Offscreen (1080p) M Texels/s 76 3291 34507 39776
Fill rate - Onscreen M Texels/s 60 2324 8760 7949
GFXBench 2.5 Egypt HD C24Z16 - Offscreen (1080p) Frames 124 4274 6610 70733
GFXBench 2.5 Egypt HD C24Z16 - Offscreen (1080p) Frames / s 1.1 76.3 58.5 626
GFXBench 2.5 Egypt HD C24Z16 - Onscreen Frames 74 2622 4682 6776
GFXBench 2.5 Egypt HD C24Z16 - Onscreen Frames / s 0.7 46.8 41.4 60
GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p) Frames 26 3124 3719 24434
GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p) Frames / s 0.5 55.8 66.4 436.3
GFXBench 2.7 T-Rex HD C24Z16 - Onscreen Frames 15 4274 2708 3358
GFXBench 2.7 T-Rex HD C24Z16 - Onscreen Frames / s 0.3 76.3 48.4 60
GFXBench 2.7 T-Rex HD Fixed Timestep - Offscreen (1080p) ms 1199830 56022 9117 1546
GFXBench 2.7 T-Rex HD Fixed Timestep - Offscreen (1080p) Frames / s 0.5 56022 61.4 1546
GFXBench 2.7 T-Rex HD Fixed Timestep - Onscreen ms 1699985 10 11734 362.2
GFXBench 2.7 T-Rex HD Fixed Timestep - Onscreen Frames / s 0.3 6761 47.7 9361
Triangle throughput: Textured - Offscreen (1080p) M Triangles/s 1 59.8 2209 59.8
Triangle throughput: Textured - Onscreen M Triangles/s 0 303 6246 2098
Triangle throughput: Textured, fragment lit - Offscreen (1080p) M Triangles/s 1 300 1770 2049
Triangle throughput: Textured, fragment lit - Onscreen M Triangles/s 0 303 4263 1678
Triangle throughput: Textured, vertex lit - Offscreen (1080p) M Triangles/s 1 299 1780 1665
Triangle throughput: Textured, vertex lit - Onscreen M Triangles/s 0 300 7310 1672
Host: Microsoft Windows Server 2012 Datacenter Edition,
Guest: Microsoft Windows Server 2012 Standard Evaluation,
Client: FreeRDP on Mac OSX * http://gfxbench.com/

9. Page 9
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
XenServer with VNC / RDP / Guacamole
 Technology
 GPU pass-through of Nvidia GRID K2 to Windows and Linux guests.
 Two GPU devices available for two separate instances.
 Physical GPU virtualization drivers promised by end of the year (n=64?).
 Host / Hypervisor
 XenServer 6.1 Platinum Edition (enables GPU license).
 Guests
 Linux: specify Windows 7 as server template then standard CentOS 6.4
install process. Good GPU performance using VNC and VirtualGL.
 Windows: tried both RDP and TigerVNC do software OpenGL rendering.
• Unable to VNC or RDP to “console” display, possibly no monitor?
 Clients
 Windows and OSX Clients comparable to remote desktop to native boxes.
 Tried RDP, VNC, and Guacamole (HTML5 VNC client).
 OpenStack Status
 XenServer Integrates with OpenStack using XenServer plugins.
 Nova scheduler support for XenServer not on blueprint radar.

10. Page 10
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Open Source Libvirt/Xen with VNC / RDP / Guacamole
 Technology
 GPU pass-through of Nvidia GRID K2 to Windows and Linux guests.
 Two GPU devices available for two separate instances.
 Unsure of GPU virtualization drivers status for n > 2.
 Host / Hypervisor
 Ubuntu 12.04, Xen (Xen hypervisor 4.1.0), libvirt (0.98)
 Xen HVM guest cfgfile in /etc/xen/<guest> to enable VGA passthrough:
gfx_passthru=1
pci=['yy:zz.n’]
 Guests
 Linux: Tested CentOS 6.4 image with latest Nvidia drivers (310.44).
Verified device functioned. Expect similar results to XenServer.
 Windows: Did not test an image. Expect similar results to XenServer.
 OpenStack Status
 Xen supported within OpenStack using nova libvirt driver.
 Nova specs support for libvirt exists (i.e. this host has a GPU).
 Feasible to have support included in H or I release given lxc work.
http://wiki.xen.org/wiki/Xen_VGA_Passthrough

11. Page 11
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
XenServer Linux Virtual Machine Installation Hack

12. Page 12
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Specify GPU Type

13. Page 13
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
VirtualGL Architecture (VirtualGL.org)

14. Page 14
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
GLXINFO for Linux Instance on XenServer

15. Page 15
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Guacamole HTML5 Browser VNC

16. Page 16
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Windows Server 2012 on XenServer

17. Page 17
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Heterogeneous Architecture Support Blueprints
https://wiki.openstack.org/wiki/Heter
ogeneousInstanceTypes
https://wiki.openstack.org/wiki/Sc
heduleHeterogeneousInstances
https://wiki.openstack.org/wiki/Heterog
eneousGpuAcceleratorSupport

18. Page 18
GPU-Accelerated VDI on OpenStack
NIMBIS SERVICES, INC.
Summary
 GPU-accelerated VDI pools within OpenStack are quite feasible.
 Code support is very rough around the edges, not production ready.
 We really need to get open community support organized for this feature.
 Several vertical vendor solutions out there, but Xen and KVM baselines
should be possible as open-source.
 Questions?