GPU-Virtualization-in-openSUSE.pdf

•

0 likes•54 views

This document discusses GPU virtualization in openSUSE. It covers: 1. An overview of GPU virtualization, its use cases and market potential. 2. The critical techniques used, including SRIOV vs MDEV approaches. 3. A demo of GPU virtualization in action. 4. The current status and future plans, including support from vendors like Nvidia, AMD and Intel.

Technology

GPU virtualization
in openSUSE
Virtualization Team
Liang Yan
lyan@suse.com

GPU virtualization in openSUSE
1. GPU virtualization
Definition and Classification
Use scenario
2. Critical techniques
SRIOV vs MDEV
3. Demo
4. Current status and Todo
openSUSE
Outside Vendors
5. Q&A

GPU Virtualization
Huge market: Gartner Report:
2017: $ 145B
2018: $ 175B
2019: $ 206B
User Cases:
Auto driver: Tesla
Medical area:
Finance: wall street
Electronic Commerce: Delivery Transport, Recommend Sale
Language Translate/ AI:

GPU Virtualization
GPU Virtualization in Cloud, providing machine
learning service
Google Colaboratory
Paperspace Gradient
FloydHub Workspace
Lambda GPU Cloud
AWS Deep Learning AMIs
GCP Deep Learning VM Images

GPU Virtualization
GPU virtualization, Software Implementation
● Software Emulated
CPU “trap and emulate” GPU instruction,Slow, limited function
● API forwarding
Intercept API, Simple idea, but painful for API compatible
IO virtualization, GPU as a PCIe device today.
● GPU Passthrough
● Full GPU Virtualization
Para?
VirtIO-GPU

Run “native” graphics driver in VM, Full API, 3D
Achieve good performance and moderate multiplexing capability
● Split
Time Slices
framebuffer memory
● Isolate
Give a neat access between VM and Host Physical Device
IOMMU/Mdev and VFIO
DMA
Interrupt
● Schedule
Efficient and Robust
Pretty fix for AMD, hardware implementation
More flexible for NVIDIA, RR, BOND
GPU Virtualization

GPU Virtualization
Nvidia
Tesla Series: Volta Pascal Maxwell M6 M10 M60 P4 P6 P40 P100 V100
GRID: Kepler K1 K2 (VDI and application virtualization)
http://www.nvidia.com/object/grid-certified-servers.html
AMD
FirePro S7150 S7150x2
Radeon Pro V320 V340
Radeon Instinct MI6 MI8 MI25(Machine learning interface, CUDA compatible with HIP)
MI50/60 MI100
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004075.html
Intel
Haswell(3VMs) Broadwell(7VMs) Skylake, Kaby Lake
2020, dedicated GPU
https://github.com/intel/gvt-linux/wiki

SRIOV vs Mdev
GPU real Workload:
This is why GPU virtualization is doable

10
MDEV 80~90%
no BSF, UUID
No IOMMU, MMIO trap
vGPU command needs address translation
Time Slice
Could split more vGPU
VFIO-Mdev Passthrough
SRIOV 97%
Bus/Slot/Function
IOMMU/VFIO
vGPU command runs directly
Time Slice
VF number is fixed
VFIO Passthrough
SR-IOV Mediated

MEDIATED DEVICE FRAMEWORK
Physical Interface sysfs Mdev Bus
Vendor Mdev
Vfio-mdev.ko
Vfio API compatible
TYPE1 IOMMU
i915.ko
nvidia.ko
Management
create
destroy

SUSE
- Intel KVMGT technical ready
- Nvidia GRID technical ready
- AMD MxGPU technical ready
- GPU virtualization for Cloud
- GPU virtualization for CAAS
Outside
- Remote display
- IOMMU compatible
- Live Migration
- Scalability: Schedule Algorithm **
GPU for Container
FPGA/NPU/TPU Virtualization?
Status and Future

REFERENCE
VGPU ON KVM
An Introduction to PCI Device Assignment with VFIO - Alex Williamson,
Red Hat [Qemu-devel] [PATCH v7 0/4] Add Mediated device support
[libvirt] [RFC] libvirt vGPU QEMU integratio
https://yq.aliyun.com/articles/590909?spm=a2c4e.11153940.blogcont599189.23.f2016d7bXPo7TD
https://zhuanlan.zhihu.com/p/35489035

2016年7月22日（金）のNVIDIA Deep Learning Day 2016 Summer の講演資料です。エヌビディア合同会社技術顧問馬路徹 NVIDIA DRIVE PX2 は高度な画像認識等を可能とするディープラーニング及び自動運転の主要な機能を実現する DriveWorks SDK を実装した車載組込み用プラットフォームです。本講演では NVIDIA DRIVE PX2 の概要と画像認識、自動運転等の応用例もご紹介します。

GTC 2022 Keynote

Alison B. Lowndes

This document discusses NVIDIA's technologies for artificial intelligence and accelerated computing. It highlights NVIDIA's GPUs, systems, SDKs, and frameworks that power AI workloads at scale. These include the H100 GPU, DGX systems, Triton inference server, RAPIDS libraries, and Omniverse platform for simulation and digital twins. The document also outlines key applications and industries that are being accelerated by NVIDIA's technologies like autonomous vehicles, healthcare, robotics, and more.

Part 2 Maximizing the utilization of GPU resources on-premise and in the cloud

Univa, an Altair Company

This document discusses NVIDIA's updates from spring through fall 2020. It introduces the new NVIDIA AMPERE architecture with the GeForce RTX 30 series GPUs. It also discusses the release of the NVIDIA HPC SDK and NVIDIA GTC fall 2020 conference. The document focuses on NVIDIA's role in accelerating AI workloads and introduces the new DGX SuperPod AI infrastructure solution. It outlines challenges with deploying AI on GPUs and proposes solutions around provisioning specialized GPU infrastructure and monitoring GPU utilization.

GPU Accelerated Virtual Desktop Infrastructure (VDI) on OpenStack

Brian Schott

Backend.AI Technical Introduction (19.09 / 2019 Autumn)

Lablup Inc.

VMworld 2015: Deliver High Performance Desktops with VMware Horizon and NVIDI...

VMworld

This presentation discusses delivering high performance desktops using VMware Horizon and NVIDIA GRID vGPU. It includes a demo and overview of understanding graphics use cases. It discusses using Horizon 6 and GRID vGPU for graphics, NVIDIA GRID and announcing NVIDIA GRID 2.0. It covers achieving the pinnacle of performance and "60 in 60" which is about getting a proof of concept up and running quickly. It also discusses Dell hardware options and includes resources and Q&A. The presentation is aimed at an audience interested in virtual desktop infrastructure and high performance graphics solutions.

This document discusses the author's journey doing AI research in the cloud using open-source tools. It provides an overview of AI concepts and frameworks like PyTorch. It then describes a project called Habitat that predicts deep learning training times in the cloud. Next, it discusses AI accelerators like GPUs, distributed training, and considerations for AI cloud implementation and providers. The author shares lessons learned around dependencies, costs, and setup experiences with various cloud platforms.

GPU Cloud Server in India

Cloudtechtiq

How our Cloudy Mindsets Approached Physical Routers

Steffen Gebert

The document discusses how EMnify integrated a pair of Juniper routers into their existing cloud-based workflows and monitoring tools. They deployed the routers using Ansible playbooks for configuration management and leveraged existing tools like Prometheus, Grafana, and CloudWatch for monitoring metrics, logs, and alerts. While the integration worked well, they note some challenges around testing configurations and limitations of the monitoring tools for high data volumes. The overall approach focused on minimizing new processes and tools by bridging the routers into their existing cloud-centric tooling.

Harnessing the virtual realm for successful real world artificial intelligence

Alison B. Lowndes

Artificial Intelligence is impacting all areas of society, from healthcare and transportation to smart cities and energy. How NVIDIA invests both in internal pure research and accelerated computation to enable its diverse customer base, across gaming & extended reality, graphics, AI, robotics, simulation, high performance scientific computing, healthcare & more. You will be introduced to the GPU computing platform & shown real world successfully deployed applications as well as a glimpse into the current state of the art across academia, enterprise and startups.

GPU-Accelerating UDFs in PySpark with Numba and PyGDF

Keith Kraus

This document discusses accelerating Python user-defined functions (UDFs) in PySpark using Numba and PyGDF. It describes how data movement between the JVM and Python workers is currently a bottleneck for PySpark Python UDFs. With Apache Arrow, data can be transferred in a columnar format without serialization, improving performance. PyGDF enables defining UDFs that operate directly on GPU data frames using Numba for further acceleration. This allows leveraging GPUs to optimize complex UDFs in PySpark. Future work includes optimizing joins in PyGDF and supporting distributed GPU processing.

08 - it3D Summit 2016 - Grid - T. Riley- NVIDIA

Virginia Grubert

ENTER NVIDIA GRID Delivering accelerated virtual desktops and applications. This is where NVIDIA, the leader in graphics acceleration, stepped in to help. NVIDIA GRID technology allows IT to virtualize the physical GPU sitting in a server and share it with multiple VDI instances. This means that IT can deliver a true PC experience to any remote device from the datacenter. By providing a way to bring graphics acceleration to virtualization, NVIDIA GRID allows you to unlock all of the promises of productivity, mobility, security and flexibility for every one of your users. With NVIDIA GRID you can safely house ALL your current into the datacenter so that they can be delivered out to any device, be it a thin client, chromebook, iPad or BYO Device. From an end user perspective this means they can be more productive working with the devices and in the locations that best suit them. For IT they are able to manage everything in a centrally in the datacenter which vastly simplifies their life. by Mr. Tom Riley, Director Global Business Development - Enterprise VR

Metaverse - The 'Killer App' for 5G, 6G and Beyond

Anand Bhojan

"Metaverse - The 'Killer App' for 5G, 6G and Beyond": With the growing popularity of X-Reality (VR/AR/MR/Metaverse) generation of global media-data is expected to accelerate at much higher rate and they will drive innovations and advancements in mobile networks. In this talk, challenges in Metaverse and we will explore suitable system architectures for streaming and analytics for Metaverse with the 5G edge nodes.

20201006_PGconf_Online_Large_Data_Processing

Kohei KaiGai

PG-Strom is an extension of PostgreSQL that utilizes GPUs and NVMe SSDs to enable terabyte-scale data processing and in-database analytics. It features SSD-to-GPU Direct SQL, which loads data directly from NVMe SSDs to GPUs using RDMA, bypassing CPU and RAM. This improves query performance by reducing I/O traffic over the PCIe bus. PG-Strom also uses Apache Arrow columnar storage format to further boost performance by transferring only referenced columns and enabling vector processing on GPUs. Benchmark results show PG-Strom can process over a billion rows per second on a simple 1U server configuration with an NVIDIA GPU and multiple NVMe SSDs.

Hardware in Space

Alison B. Lowndes

This document provides a summary of announcements and information from NVIDIA regarding their AI products and technologies. It highlights the release of the NVIDIA A100 80GB GPU, the Selene supercomputer featuring DGX A100 systems, comparisons of AI data center configurations with DGX-1 versus DGX A100 systems, and expansions to the NGC container environment and RAPIDS open source libraries. Brief descriptions of applications like SIMNET and Fourier Neural Operators are also included.

NVIDIA PRO VR DAY 2017 基調講演

NVIDIA Japan

The document provides an overview of NVIDIA's professional VR solutions and technologies. It discusses the computing challenges of reproducing reality in VR/AR, including graphics/display, audio, touch/physics, and capturing 360 video. It highlights NVIDIA's VRWorks toolkit and Quadro VR solutions that help address these challenges. Key applications of professional VR discussed include design, manufacturing, medical, and collaboration workflows.

PGI Compilers & Tools Update- March 2018

NVIDIA

This document provides an update on PGI compilers and tools for heterogeneous supercomputing. It discusses PGI's support for OpenACC directives to accelerate applications on multicore CPUs and NVIDIA GPUs from a single source. It highlights new compiler features including support for Intel Skylake, AMD EPYC and IBM POWER9 CPUs as well as NVIDIA Volta GPUs. Benchmark results show strong performance of OpenACC applications on these platforms. The document also discusses the growing adoption of OpenACC in HPC applications and resources available to support OpenACC development.

Introducing Amazon EC2 P3 Instance - Featuring the Most Powerful GPU for Mach...

Amazon Web Services

Amazon EC2 P3 instances offer up to eight of the latest NVIDIA Tesla V100 GPUs, with up to 13X the speed of previous generation GPU instances. In this session, learn from Airbnb how they use machine learning to make their services smarter and more engaging for their customers and how they are using P3 instances to dramatically lower training time of their machine learning models while optimize costs.

Kubernetes Native Infrastructure and CoreOS Operator Framework for 5G Edge Cl...

Hidetsugu Sugiyama

Transparent GPU Exploitation for Java

Kazuaki Ishizaki

This document summarizes Kazuaki Ishizaki's keynote presentation at the Fourth International Symposium on Computing and Networking (CANDAR'16) on transparent GPU exploitation for Java. The presentation covered Ishizaki's research history developing compilers and optimizing code for GPUs. It described a Java just-in-time compiler that can generate optimized GPU code from parallel loops in Java programs without requiring programmers to manage low-level GPU operations like data transfers and memory allocation themselves. The compiler implements optimizations like array alignment, read-only caching, and reducing data copying to improve GPU performance. The goal is to make GPU programming easier and more portable across hardware for Java programmers.

GTC Taiwan 2017 在 Google Cloud 當中使用 GPU 進行效能最佳化

NVIDIA Taiwan

The document discusses using GPUs on Google Cloud Platform for accelerating compute-intensive workloads. It describes how GPUs can provide significant performance gains for machine learning, high performance computing, and visualization workloads. It provides examples of customers like Schlumberger leveraging GPUs on GCP for oil exploration and Shazam for music fingerprinting. The document also highlights the flexibility, scalability, and cost benefits of using GPUs on Google Cloud Platform.

Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...

Intel® Software

Monitoring of GPU Usage with Tensorflow Models Using Prometheus

Databricks

Understanding the dynamics of GPU utilization and workloads in containerized systems is critical to creating efficient software systems. We create a set of dashboards to monitor and evaluate GPU performance in the context of TensorFlow. We monitor performance in real time to gain insight into GPU load, GPU memory and temperature metrics in a Kubernetes GPU enabled system. Visualizing TensorFlow training job metrics in real time using Prometheus allows us to tune and optimize GPU usage. Also, because Tensor flow jobs can have both GPU and CPU implementations it is useful to view detailed real time performance data from each implementation and choose the best implementation. To illustrate our system, we will show a live demo gathering and visualizing GPU metrics on a GPU enabled Kubernetes cluster with Prometheus and Grafana.

Graphics processing unit ppt

Sandeep Singh

The document discusses the evolution of GPU architecture and capabilities over time. It describes how GPUs have become massively parallel processors with programmable capabilities beyond just graphics. The document outlines the core components of a GPU including the graphics pipeline and programming model. It also discusses how GPUs are well suited for parallel, data-intensive applications and how their capabilities have expanded into general purpose computing through technologies like CUDA.

Deep Dive on Amazon EC2 Accelerated Computing

Amazon Web Services

by Jeanine Banks, Director of Product Management, EC2 Windows & Enterprise Workloads, AWS Researchers, scientists and IT organizations are looking to develop, deploy and deliver machine learning and HPC workloads by leveraging the agility, scalability and availability of the public cloud. Amazon EC2 Accelerated Computing platform products include Amazon EC2 P3 instances, Amazon EC2 G3 instances and Amazon EC2 F1 instances. This session will provide a detailed technical deep dive of the Amazon EC2 Accelerated Computing platforms, which are Amazon EC2 P3, Amazon EC2 G3 and Amazon EC2 F1 instances, their key market use cases which include machine learning, high performance computing, scientific research and reconfigurable computing.

Stable-Diffusion-v2.pdf

GPU-Virtualization-in-openSUSE.pdf

Recommended

Recommended

More Related Content

Similar to GPU-Virtualization-in-openSUSE.pdf

Similar to GPU-Virtualization-in-openSUSE.pdf (20)

More from Liang Yan

More from Liang Yan (12)

Recently uploaded

Recently uploaded (20)

GPU-Virtualization-in-openSUSE.pdf