Presentation delivered at the 2017 LinuxCon China. Container is a popular technology in cloud with the merits of fast provisioning, high density and near-native performance. New requirements have been rising to further use GPU to accelerate various applications in containers, e.g. media trans-coding, machine learning, etc. However there is still gap of managing GPU in such container usages. This presentation will first review the status of GPU support on both native container and Intel clear container, then provide an anatomy of technical gaps and enabling plans in major areas (orchestration layer, resource isolation and QoS performance isolation), then review our work for GPU virtualization in clear container and prototype work through extensions in docker plugin, cgroup and GPU scheduler to fix those gaps.

1. 1
GPU Acceleration for Container on Intel
Processor Graphics
Zhenyu Wang
zhenyuw@linux.intel.com

2. 2
Agenda
GPU Containers
GPU Namespace
GPU Control Group
Integration with Container Runtime
VM Containers
Status

3. 3
GPU Containers
Video Delivery
(Encode/Decode/Transcode)
Cloud Graphics
(Remote desktop/gaming)
HPC/Machine Learning
(AI, Analytic, …)
E3-G Discrete
GPU
Fast provisioning
High Density
Little Overhead

4. 4
What Makes a Container?
Platform
Host OS
Namespaces
Control
groups
App
Libs
App
Libs
App
Libs
Platform
Hypervisor
App
Libs
App
Libs
Guest OS
App
Libs
Guest OS
Bare Metal Containers
Docker DockerDocker
VM Containers
(e.g. Clear Container)

5. 5
Path to GPU Containers
Bare Metal Containers VM Containers
GPU namespaces
GPU control groups
GPU virtualization technology
Integrate with container runtime

6. 6
GPU Namespace:
DRM Render Node
DRM/i915
Card# RenderD#
Xserver ComputeMedia
Xcode
Unprivileged
No mode-setting
Offscreen
3D
Privileged
Mode setting
Global gem flink

7. 7
GPU Namespace:
DRM Render Node
DRM/i915
RenderD#
Media
Xcode
Libs
Offscreen
3D
Libs
Compute
Libs
GPU
Contexts
Sharing happen with dmabuf only (no
global gem object on card0)
No need of creating another namespace
Expose DRM render nodes to provide
isolated views through device cgroup
 e.g “docker run –device=/dev/dri/renderD128
–it debian”
Per-process hardware context in GPU
 Hardware managed render context switch

8. 8
GPU CGroup
CPU Mem BlkIO Net GPU
container
Container
resource
manager
Cgroup
controller
container container container
…
Container-granule GPU resource control

9. 9
GPU CGroup
New subsystem: gpu_cgrp_subsys
Control knobs
 gpu.shares (% share of GPU cycles, work in progress)
 gpu.priority (workload priority in the system)
 gpu.memory (maximum GPU memory size)
DRM/i915
 Favor ‘shares/priority’ in workload scheduler
 Enforce memory limitation (optionally for UMA graphics)

10. 10
Container Runtime Integration
{
“linux”: {
“devices”: [
{
“path”: “/dev/dri/renderD128”,
“type”: “c”,
“major”: 226,
“minor”: 128,
“fileMode”: 432,
“uid”: 0,
“gid”: 0
}
],
“resources”: {
…
“gpu”: {
“memory”: <max_mem_in_bytes>,
“prio”: <workload_priority>
}
}
}
RunC: an universal container runtime
 Understand new gpu cgroup
 Add new Linux resources config.json options for gpu
 Runtime-tools helper to generate config stubs
New Docker GPU control parameters
e.g docker –gpu-mem=xxx –gpu-priority=xxx
–gpu-share=xxx …

11. 11
Image Portability
Main challenge - library compatibility
 User level libraries MUST match underlying kernel/HW
Introduce Docker helper for image inspection
 Compare image libraries with host environment
 Reject to launch container upon any mismatch

12. 12
VM GPU Containers
DRM/i915
vGPU
Media
Xcode
Libs
GPU
vGPU vGPU
Intel GVT-g
Guest OS
Media
Xcode
Libs
Guest OS
Media
Xcode
Libs
Guest OS
CC VM CC VM CC VM Clear Container (CC)
Intel graphics virtualization technology
(Intel GVT-g)
Assign vGPU to VM container
In upstream Linux since v4.10
https://01.org/igvt-g

13. 13
VM GPU Containers
Nested containers in VM also have GPU access
Need Improvement:
Scalability
 Only support 8vGPUs today due to resource partitioning
 Need some enlightened way to further scale in the short term
Boot time
 Full guest graphics driver load takes >1.5s
 Fast optimization to <0.5s in progress

14. 14
Status
 GPU cgroup PoC
https://github.com/zhenyw/linux
https://github.com/zhenyw/runc
https://github.com/zhenyw/moby
https://github.com/zhenyw/cli
 GPU virtualization for Clear container
https://clearlinux.org/features/intel%C2%AE-clear-containers
https://github.com/01org/gvt-linux/wiki/Clear-Container-with-GVTg-Setup-Guide

15. Q & A