GPU Accelerated Virtual Desktop Infrastructure (VDI) on OpenStack

4,878 views

Published on

This is a talk presented at the OpenStack DC Meetup #9 of GPU pass-through of an Nvidia GRID K2 card with XenServer, Microsoft Hyper-V, and open source Xen hypervisors. We looked at

Published in: Technology, Art & Photos
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,878
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
111
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

GPU Accelerated Virtual Desktop Infrastructure (VDI) on OpenStack

  1. 1. Page 1GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.GPU-Accelerated VDI on OpenStackOpenStack DC MeetUpJune 18, 2013Nimbis Services, Inc.http://www.nimbisservices.comBrian Schott, CTOBrian.Schott@NimbisServices.com
  2. 2. Page 2GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.About Nimbis ServicesNimbis Services is lowering the barriers to high-performance computing (HPC) and offering a securemarketplace for technical computing in the cloud.Nimbis partners with leading HPC & cloud providers, softwarevendors, and domain experts to configure and broker secureon-demand pay-as-you-go access to applications.
  3. 3. Page 3GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Challenges for GPU-Accelerated Virtual Desktop Infrastructure (VDI)on OpenStack Making the GPU available to a virtual machine.(3 basic approaches, ok, maybe 4). Bare Metal / Containers (lxc) PCI Pass-Through Para-virtualization (proxy device drivers) Remote Display Protocols (X, NX, VirtualGL, WebGL) Remote desktop protocol that supports GPU acceleration. Host and Guest Combinations: Linux or Windows Remote Clients: Linux, Windows, OSX, Web Browser, Tablet Networking: “The Joys of Firewalls”, or “Any Port as Long as it is 443” Licenses: “This is too good to be free.” Provisioning GPU resources from within OpenStack. Nova compute virtualization driver (plugin). GPU instance types, resource discovery, and scheduling. This is still under development in the community. Full disclosure: Nimbis hand-deployed the GPU instances for testing.
  4. 4. Page 4GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Nimbis Testbed Hardware “Production” Cisco System (1) UCS C220 M3 – dual quad-core 3.3GHzIntel Xeon E5-2643, 192GB SDRAM, 2TBSAS-7K, dual channel 10GbE interface. (2) UCS C220 M3 – dual quad-core 2.4GHzIntel Xeon E5-2609, 192GB SDRAM, 2TBSAS-7K, dual channel 10GbE interface. (3) UCS C240 M3 – dual quad-core 2.4GHzIntel Xeon E5-2609, 192GB SDRAM, 6TBSAS-7K, dual-channel 10GbE.• (1) Nvidia GRID K2 Board (2) Nexus 3064 10GbE Switches (2) ASA 5520 Adaptive Security Appliances (2) Cisco Catalyst 2960S 1GbE switches “Development” Cisco System (3) Cisco UCS C200 M2 (1) Nexus 3064 10GbE Switch (1) ASA 5520 Adaptive Security Appliance (1) Cisco Catalyst 2960S 1GbE switchCores: 72Memory: 1440 GBDisk: 36 TBServers: 9*Switches: 9** What do youexpect with aCisco testbed?
  5. 5. Page 5GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Nvidia GRID GPU Virtualization (Kepler Architecture)Source: http://www.nvidia.com/object/grid-boards.htmlSource: 1http://www.nvidia.com/content/PDF/kepler/NV_DS_Tesla_KCompute_Arch_May_2012_LR.pdfGPU Virtualization¹GRID boards featurethe NVIDIA Keplerarchitecture that, for thefirst time, allowshardware virtualizationof the GPU. Thismeans multiple userscan share a single GPU,improving user densitywhile providing truePC performance andcompatibility.
  6. 6. Page 6GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.GPU-Accelerated VDI Approaches Considered Customer Requirements Combination of Windows and Linux guest images. Applications using OpenGL on both Windows and Linux. Variety of Windows (95, 7, 8) clients, OSX clients, and web browsers(IE7, IE8, Firefox, Chrome, Safari). Variety of firewall and IT/FSO software challenges. Technologies Tested Microsoft Hyper-V and RemoteFX XenServer with VNC / RDP / Guacamole Open Source Xen (libvirt) on Ubuntu Linux with VNC Possible Alternatives Linux Containers (lxc)• USC/ISI team has the blueprint and code targeted for Havana.• No Windows support. XenDesktop with HDX 3D Pro (not tested)• Requires a custom client, Citrix Receiver, OpenGL support unclear.• Does not integrate with OpenStack.
  7. 7. Page 7GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Microsoft Hyper-V and RemoteFX Technology Windows Host GPU device with para-virtualized driver for windows guests. Host / Hypervisor Microsoft Windows Server 2012 Datacenter Edition (RemoteFX license). Latest Nvidia driver for Windows. Guests No GPU pass-through available or driver for Linux guests. Accelerates Windows DirectX calls, does software OpenGL rendering. Clients Good performance with standard Windows RDC > 8.0. Good performance from FreeRDP* OSX client, not Microsoft client. iOS and Android? Not tested, but FreeRDP clients exist on both. OpenStack Status Integrates with OpenStack Hyper-V nova compute drivers. Nova scheduler support for Hyper-V extra specs not on blueprint radar. Possible to use today with custom nova scheduler plugin.* http://www.freerdp.com
  8. 8. Page 8GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Hyper-V and RemoteFX Benchmarks with Nvidia GRID K2GFXBench Test MetricRDP WithoutRemoteFXNVidiaQuadro 600(~$150)*RDP WithRemoteFXNVidiaQuadro K5000(~$1800)*Spark LinesFill rate - Offscreen (1080p) M Texels/s 76 3291 34507 39776Fill rate - Onscreen M Texels/s 60 2324 8760 7949GFXBench 2.5 Egypt HD C24Z16 - Offscreen (1080p) Frames 124 4274 6610 70733GFXBench 2.5 Egypt HD C24Z16 - Offscreen (1080p) Frames / s 1.1 76.3 58.5 626GFXBench 2.5 Egypt HD C24Z16 - Onscreen Frames 74 2622 4682 6776GFXBench 2.5 Egypt HD C24Z16 - Onscreen Frames / s 0.7 46.8 41.4 60GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p) Frames 26 3124 3719 24434GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p) Frames / s 0.5 55.8 66.4 436.3GFXBench 2.7 T-Rex HD C24Z16 - Onscreen Frames 15 4274 2708 3358GFXBench 2.7 T-Rex HD C24Z16 - Onscreen Frames / s 0.3 76.3 48.4 60GFXBench 2.7 T-Rex HD Fixed Timestep - Offscreen (1080p) ms 1199830 56022 9117 1546GFXBench 2.7 T-Rex HD Fixed Timestep - Offscreen (1080p) Frames / s 0.5 56022 61.4 1546GFXBench 2.7 T-Rex HD Fixed Timestep - Onscreen ms 1699985 10 11734 362.2GFXBench 2.7 T-Rex HD Fixed Timestep - Onscreen Frames / s 0.3 6761 47.7 9361Triangle throughput: Textured - Offscreen (1080p) M Triangles/s 1 59.8 2209 59.8Triangle throughput: Textured - Onscreen M Triangles/s 0 303 6246 2098Triangle throughput: Textured, fragment lit - Offscreen (1080p) M Triangles/s 1 300 1770 2049Triangle throughput: Textured, fragment lit - Onscreen M Triangles/s 0 303 4263 1678Triangle throughput: Textured, vertex lit - Offscreen (1080p) M Triangles/s 1 299 1780 1665Triangle throughput: Textured, vertex lit - Onscreen M Triangles/s 0 300 7310 1672Host: Microsoft Windows Server 2012 Datacenter Edition,Guest: Microsoft Windows Server 2012 Standard Evaluation,Client: FreeRDP on Mac OSX * http://gfxbench.com/
  9. 9. Page 9GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.XenServer with VNC / RDP / Guacamole Technology GPU pass-through of Nvidia GRID K2 to Windows and Linux guests. Two GPU devices available for two separate instances. Physical GPU virtualization drivers promised by end of the year (n=64?). Host / Hypervisor XenServer 6.1 Platinum Edition (enables GPU license). Guests Linux: specify Windows 7 as server template then standard CentOS 6.4install process. Good GPU performance using VNC and VirtualGL. Windows: tried both RDP and TigerVNC do software OpenGL rendering.• Unable to VNC or RDP to “console” display, possibly no monitor? Clients Windows and OSX Clients comparable to remote desktop to native boxes. Tried RDP, VNC, and Guacamole (HTML5 VNC client). OpenStack Status XenServer Integrates with OpenStack using XenServer plugins. Nova scheduler support for XenServer not on blueprint radar.
  10. 10. Page 10GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Open Source Libvirt/Xen with VNC / RDP / Guacamole Technology GPU pass-through of Nvidia GRID K2 to Windows and Linux guests. Two GPU devices available for two separate instances. Unsure of GPU virtualization drivers status for n > 2. Host / Hypervisor Ubuntu 12.04, Xen (Xen hypervisor 4.1.0), libvirt (0.98) Xen HVM guest cfgfile in /etc/xen/<guest> to enable VGA passthrough:gfx_passthru=1pci=[yy:zz.n’] Guests Linux: Tested CentOS 6.4 image with latest Nvidia drivers (310.44).Verified device functioned. Expect similar results to XenServer. Windows: Did not test an image. Expect similar results to XenServer. OpenStack Status Xen supported within OpenStack using nova libvirt driver. Nova specs support for libvirt exists (i.e. this host has a GPU). Feasible to have support included in H or I release given lxc work.http://wiki.xen.org/wiki/Xen_VGA_Passthrough
  11. 11. Page 11GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.XenServer Linux Virtual Machine Installation Hack
  12. 12. Page 12GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Specify GPU Type
  13. 13. Page 13GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.VirtualGL Architecture (VirtualGL.org)
  14. 14. Page 14GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.GLXINFO for Linux Instance on XenServer
  15. 15. Page 15GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Guacamole HTML5 Browser VNC
  16. 16. Page 16GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Windows Server 2012 on XenServer
  17. 17. Page 17GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Heterogeneous Architecture Support Blueprintshttps://wiki.openstack.org/wiki/HeterogeneousInstanceTypeshttps://wiki.openstack.org/wiki/ScheduleHeterogeneousInstanceshttps://wiki.openstack.org/wiki/HeterogeneousGpuAcceleratorSupport
  18. 18. Page 18GPU-Accelerated VDI on OpenStackNIMBIS SERVICES, INC.Summary GPU-accelerated VDI pools within OpenStack are quite feasible. Code support is very rough around the edges, not production ready. We really need to get open community support organized for this feature. Several vertical vendor solutions out there, but Xen and KVM baselinesshould be possible as open-source. Questions?

×