GPU Accelerated Virtual Desktop Infrastructure (VDI) on OpenStack
Page 1GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.GPU-Accelerated VDI on OpenStackOpenStack DC MeetUpJune 18, 2013Nimbis Services, Inc.http://www.nimbisservices.comBrian Schott, CTOBrian.Schott@NimbisServices.com
Page 2GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.About Nimbis ServicesNimbis Services is lowering the barriers to high-performance computing (HPC) and offering a securemarketplace for technical computing in the cloud.Nimbis partners with leading HPC & cloud providers, softwarevendors, and domain experts to configure and broker secureon-demand pay-as-you-go access to applications.
Page 3GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Challenges for GPU-Accelerated Virtual Desktop Infrastructure (VDI)on OpenStack Making the GPU available to a virtual machine.(3 basic approaches, ok, maybe 4). Bare Metal / Containers (lxc) PCI Pass-Through Para-virtualization (proxy device drivers) Remote Display Protocols (X, NX, VirtualGL, WebGL) Remote desktop protocol that supports GPU acceleration. Host and Guest Combinations: Linux or Windows Remote Clients: Linux, Windows, OSX, Web Browser, Tablet Networking: “The Joys of Firewalls”, or “Any Port as Long as it is 443” Licenses: “This is too good to be free.” Provisioning GPU resources from within OpenStack. Nova compute virtualization driver (plugin). GPU instance types, resource discovery, and scheduling. This is still under development in the community. Full disclosure: Nimbis hand-deployed the GPU instances for testing.
Page 5GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Nvidia GRID GPU Virtualization (Kepler Architecture)Source: http://www.nvidia.com/object/grid-boards.htmlSource: 1http://www.nvidia.com/content/PDF/kepler/NV_DS_Tesla_KCompute_Arch_May_2012_LR.pdfGPU Virtualization¹GRID boards featurethe NVIDIA Keplerarchitecture that, for thefirst time, allowshardware virtualizationof the GPU. Thismeans multiple userscan share a single GPU,improving user densitywhile providing truePC performance andcompatibility.
Page 6GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.GPU-Accelerated VDI Approaches Considered Customer Requirements Combination of Windows and Linux guest images. Applications using OpenGL on both Windows and Linux. Variety of Windows (95, 7, 8) clients, OSX clients, and web browsers(IE7, IE8, Firefox, Chrome, Safari). Variety of firewall and IT/FSO software challenges. Technologies Tested Microsoft Hyper-V and RemoteFX XenServer with VNC / RDP / Guacamole Open Source Xen (libvirt) on Ubuntu Linux with VNC Possible Alternatives Linux Containers (lxc)• USC/ISI team has the blueprint and code targeted for Havana.• No Windows support. XenDesktop with HDX 3D Pro (not tested)• Requires a custom client, Citrix Receiver, OpenGL support unclear.• Does not integrate with OpenStack.
Page 7GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Microsoft Hyper-V and RemoteFX Technology Windows Host GPU device with para-virtualized driver for windows guests. Host / Hypervisor Microsoft Windows Server 2012 Datacenter Edition (RemoteFX license). Latest Nvidia driver for Windows. Guests No GPU pass-through available or driver for Linux guests. Accelerates Windows DirectX calls, does software OpenGL rendering. Clients Good performance with standard Windows RDC > 8.0. Good performance from FreeRDP* OSX client, not Microsoft client. iOS and Android? Not tested, but FreeRDP clients exist on both. OpenStack Status Integrates with OpenStack Hyper-V nova compute drivers. Nova scheduler support for Hyper-V extra specs not on blueprint radar. Possible to use today with custom nova scheduler plugin.* http://www.freerdp.com
Page 8GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Hyper-V and RemoteFX Benchmarks with Nvidia GRID K2GFXBench Test MetricRDP WithoutRemoteFXNVidiaQuadro 600(~$150)*RDP WithRemoteFXNVidiaQuadro K5000(~$1800)*Spark LinesFill rate - Offscreen (1080p) M Texels/s 76 3291 34507 39776Fill rate - Onscreen M Texels/s 60 2324 8760 7949GFXBench 2.5 Egypt HD C24Z16 - Offscreen (1080p) Frames 124 4274 6610 70733GFXBench 2.5 Egypt HD C24Z16 - Offscreen (1080p) Frames / s 1.1 76.3 58.5 626GFXBench 2.5 Egypt HD C24Z16 - Onscreen Frames 74 2622 4682 6776GFXBench 2.5 Egypt HD C24Z16 - Onscreen Frames / s 0.7 46.8 41.4 60GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p) Frames 26 3124 3719 24434GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p) Frames / s 0.5 55.8 66.4 436.3GFXBench 2.7 T-Rex HD C24Z16 - Onscreen Frames 15 4274 2708 3358GFXBench 2.7 T-Rex HD C24Z16 - Onscreen Frames / s 0.3 76.3 48.4 60GFXBench 2.7 T-Rex HD Fixed Timestep - Offscreen (1080p) ms 1199830 56022 9117 1546GFXBench 2.7 T-Rex HD Fixed Timestep - Offscreen (1080p) Frames / s 0.5 56022 61.4 1546GFXBench 2.7 T-Rex HD Fixed Timestep - Onscreen ms 1699985 10 11734 362.2GFXBench 2.7 T-Rex HD Fixed Timestep - Onscreen Frames / s 0.3 6761 47.7 9361Triangle throughput: Textured - Offscreen (1080p) M Triangles/s 1 59.8 2209 59.8Triangle throughput: Textured - Onscreen M Triangles/s 0 303 6246 2098Triangle throughput: Textured, fragment lit - Offscreen (1080p) M Triangles/s 1 300 1770 2049Triangle throughput: Textured, fragment lit - Onscreen M Triangles/s 0 303 4263 1678Triangle throughput: Textured, vertex lit - Offscreen (1080p) M Triangles/s 1 299 1780 1665Triangle throughput: Textured, vertex lit - Onscreen M Triangles/s 0 300 7310 1672Host: Microsoft Windows Server 2012 Datacenter Edition,Guest: Microsoft Windows Server 2012 Standard Evaluation,Client: FreeRDP on Mac OSX * http://gfxbench.com/
Page 9GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.XenServer with VNC / RDP / Guacamole Technology GPU pass-through of Nvidia GRID K2 to Windows and Linux guests. Two GPU devices available for two separate instances. Physical GPU virtualization drivers promised by end of the year (n=64?). Host / Hypervisor XenServer 6.1 Platinum Edition (enables GPU license). Guests Linux: specify Windows 7 as server template then standard CentOS 6.4install process. Good GPU performance using VNC and VirtualGL. Windows: tried both RDP and TigerVNC do software OpenGL rendering.• Unable to VNC or RDP to “console” display, possibly no monitor? Clients Windows and OSX Clients comparable to remote desktop to native boxes. Tried RDP, VNC, and Guacamole (HTML5 VNC client). OpenStack Status XenServer Integrates with OpenStack using XenServer plugins. Nova scheduler support for XenServer not on blueprint radar.
Page 10GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Open Source Libvirt/Xen with VNC / RDP / Guacamole Technology GPU pass-through of Nvidia GRID K2 to Windows and Linux guests. Two GPU devices available for two separate instances. Unsure of GPU virtualization drivers status for n > 2. Host / Hypervisor Ubuntu 12.04, Xen (Xen hypervisor 4.1.0), libvirt (0.98) Xen HVM guest cfgfile in /etc/xen/<guest> to enable VGA passthrough:gfx_passthru=1pci=[yy:zz.n’] Guests Linux: Tested CentOS 6.4 image with latest Nvidia drivers (310.44).Verified device functioned. Expect similar results to XenServer. Windows: Did not test an image. Expect similar results to XenServer. OpenStack Status Xen supported within OpenStack using nova libvirt driver. Nova specs support for libvirt exists (i.e. this host has a GPU). Feasible to have support included in H or I release given lxc work.http://wiki.xen.org/wiki/Xen_VGA_Passthrough
Page 11GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.XenServer Linux Virtual Machine Installation Hack
Page 12GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Specify GPU Type
Page 13GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.VirtualGL Architecture (VirtualGL.org)
Page 14GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.GLXINFO for Linux Instance on XenServer
Page 16GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Windows Server 2012 on XenServer
Page 17GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Heterogeneous Architecture Support Blueprintshttps://wiki.openstack.org/wiki/HeterogeneousInstanceTypeshttps://wiki.openstack.org/wiki/ScheduleHeterogeneousInstanceshttps://wiki.openstack.org/wiki/HeterogeneousGpuAcceleratorSupport
Page 18GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Summary GPU-accelerated VDI pools within OpenStack are quite feasible. Code support is very rough around the edges, not production ready. We really need to get open community support organized for this feature. Several vertical vendor solutions out there, but Xen and KVM baselinesshould be possible as open-source. Questions?