Learning Objectives: - Get an overview of Elastic GPUs - Dive deep on the technical capabilities of Elastic GPUs - Learn best practices when using Elastic GPUs Amazon EC2 Elastic GPUs allow you to easily attach low-cost graphics acceleration to current generation EC2 instances. With Elastic GPUs, you choose the GPU resources that are sized for your workload, so you can accelerate the graphics performance of your applications for a fraction of the cost of stand-alone graphics instances. In this tech talk, we will provide a deep dive on the capabilities of Elastic GPUs and its use case.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
David Pellerin, Business Development Principal, AWS
May 30th, 2017
Deep Dive on Amazon EC2
Elastic GPUs

2. Agenda
• Challenges in Deploying Graphics Workloads
• Overview of Amazon EC2 Elastic GPUs
• Elastic GPUs Technical Deep Dive
• How to use Elastic GPUs
• Graphics Certification Program

3. Graphics Workloads

4. Traditional Workstation

5. Application Toolchain
Data &
Process
(PLM)
Conceptual
Design
Engineering Design
(CAD)
Simulation & Analysis
(CAE)
Tooling Design
(CAM)

6. Application Toolchain On-Premises
Conceptual
Design
Application 1
Application 2
Application 3
Data Management

7. Application Toolchain On-Premises
Conceptual
Design
Application 1
Application 2
Application 3

8. Application Toolchain in AWS
Conceptual
Design
Application 1
Application 2
Application 3

9. Graphics System
Data Model Frame Display DeviceScene
CPU GPU GPU
Data Heavy Data Heavy Data Light

10. Graphics System - Remoting
Data Model Frame Display DeviceScene
CPU GPU GPU
Data Heavy Data Heavy Data Light
Generate Stream

11. Generating Graphics on AWS
• GPU-graphics instance type – G2
• Launched in November 2013
• Up to 4 NVIDIA GRID K520 GPUs per instance
• Suitable for both graphics and compute workloads
• Batch and interactive rendering of images
• Supports DirectX, OpenGL, CUDA, OpenCL

12. G2 Instances – Many Workloads Work Well
• Applications that:
• Requiring full access to GPU features
• Consume all of GPU but require only modest CPU and RAM
• That need to access hardware encoding
• Use CUDA/OpenCL together with OpenGL/DirectX
• Examples?
• Batch rendering for animation, visual effects, VR
• Interactive graphics, for example games or computer-aided
engineering (CAE)

13. However, Other Workloads Work Less Well
G2 GPU instances have fixed ratio of CPU, RAM, and
GPU, which means:
• You might be overprovisioning CPU/RAM to get the
desired amount of GPU resource
• You might be overprovisioning GPU to get more CPU
power or larger RAM

14. Elastic GPU Provides the Best of Both
Choose an EC2 instance with
the right CPU/RAM capacity
T2, M4, C4, R4, etc…
Choose a GPU with the right
graphics performance

15. Overview of Amazon EC2 Elastic GPUs
• Easily attach low-cost graphics acceleration to EC2
instances
• Choose a GPU resource sized for your workload
• Ideal If you need:
• a small amount of GPU for graphics acceleration, or
• you have applications that could benefit from some
GPU, but also require high amounts of compute,
memory, or storage

16. Overview of Amazon EC2 Elastic GPUs
• Flexible size and attachment
• Workstation-class graphics performance
• Optimized performance and cost
• OpenGL 3.0 support
Flexible Performance Cost-effective API Support

17. How Elastic GPUs Work
Application run
on CPU
3D Application run in EC2
instance, generating
OpenGL API call

18. How Elastic GPUs Work
Application run
on CPU
Elastic
GPU
Driver
OpenGL API call
intercepted by Elastic
GPU Driver
Graphics API Call

19. How Elastic GPUs Work
Application run
on CPU
Graphics API Call
Elastic
GPU
Driver
Graphics API Call
Elastic GPU driver
forwards graphics API
call to Elastic GPU
over the network

20. How Elastic GPUs Work
Application run
on CPU
Graphics API Call
Elastic
GPU
Driver
Graphics API Call
Elastic GPU processes
graphics API call and
renders image

21. How Elastic GPUs Work
Application run
on CPU
Graphics API Call
Elastic
GPU
Driver
Graphics API Call
Image
Elastic GPU encodes image
and sends it back to Elastic
GPU driver over the network

22. How Elastic GPUs Work
Application run
on CPU
Graphics API Call
Image
Elastic
GPU
Driver
Graphics API Call
Image
Elastic GPU driver receives
image and send it back to 3D
application

23. How Elastic GPUs Work
Application run
on CPU
Graphics API Call
Image
Elastic
GPU
Driver
Graphics API Call
Image

24. How Elastic GPUs Work
Application run
on CPU
Elastic
GPU
Driver
An Elastic GPU ENI (elastic
network interface) sits
between EC2 instance and
Elastic GPU
Elastic GPU is always in the
same availability zone as your
EC2 instance

25. How Elastic GPUs Work
Application run
on CPU
Elastic
GPU
Driver
This ENI is owned by
customer and managed by
AWS
Each Elastic GPU ENI
consumes 1 IPv4 address in
customer’s VPC

26. How Elastic GPUs Work
Application run
on CPU
Elastic
GPU
Driver
Elastic GPU network traffic
never leaves customer’s VPC

27. How Elastic GPUs Work
Elastic GPU does not
show up in your device
manager because it is
presented as a user
space OpenGL library

28. How Elastic GPUs Work
OpenGL applications
can detect the presence
of Elastic GPU driver
through OpenGL API call

29. How to Use Elastic GPUs
• Elastic GPU is fully integrated with EC2 Console,
AWS SDK, and AWS CLI
• At the moment, Elastic GPU only supports Windows
• Elastic GPU can be attached at instance creation,
and can be detached at instance termination

30. How to Use Elastic GPUs

31. How to Use Elastic GPUs
[ec2-user@myhost ~]$ aws ec2 run-instances --region us-east-1 --image-id ami-abcdefgh --
instance-type r4.large --key-name KEY --security-group-ids sg-12345678 --subnet-id subnet-
abcdefgh --count 1 --elastic-gpu-specification Type=eg1.xlarge
{
"OwnerId": "123456789012",
...
"ElasticGpuAssociations": [
{
"ElasticGpuAssociationState": "associating",
"ElasticGpuAssociationTime": "2017-05-19T04:22:31.000Z",
"ElasticGpuAssociationId": "egpu-assoc-09bc52c47c25cb811",
"ElasticGpuId": "egpu-f8414b807fd84ef9ace296e99ecab189"
}
],
"AmiLaunchIndex": 0
}
]
}

32. How to Use Elastic GPUs
• Elastic GPU is remotization protocol agnostic
• Works well with NICE DCV, Microsoft Remote Desktop,
VNC, any many others
NICE Desktop Cloud
Visualization (DCV)
Microsoft Remote Desktop Virtual Network
Computing (VNC)

33. How to Use Elastic GPUs
Elastic GPU UI
helps you to see
the GPU metadata
and health status
An Elastic GPU
UI icon is
available in
your notification
area

34. How to Use Elastic GPUs

35. How to Use Elastic GPUs
[ec2-user@myhost ~]$ aws ec2 describe-instances --instance-id i-042a4f99f68ebc6c5
{
"Reservations": [
{
"OwnerId": "123456789012",
...
"ElasticGpuAssociations": [
{
"ElasticGpuAssociationState": "associated",
"ElasticGpuAssociationTime": "2017-05-19T04:22:31.000Z",
"ElasticGpuAssociationId": "egpu-assoc-09bc52c47c25cb811",
"ElasticGpuId": "egpu-f8414b807fd84ef9ace296e99ecab189"
}
],
"AmiLaunchIndex": 0
}
]
}
]
}

36. How to Use Elastic GPUs
[ec2-user@myhost ~]$ aws describe-elastic-gpus --elastic-gpu-id egpu-
f8414b807fd84ef9ace296e99ecab189
{
"ElasticGpuSet": [
{
"InstanceId": "i-042a4f99f68ebc6c5",
"ElasticGpuHealth": {
"Status": "Ok"
},
"AvailabilityZone": "us-east-1b",
"ElasticGpuId": "egpu-f8414b807fd84ef9ace296e99ecab189",
"ElasticGpuType": "eg1.xlarge"
}
]
}

37. How to Use Elastic GPUs

38. Graphics Certification Program
• AWS partners with thousands of ISVs, enabling customers
to run a wide range of applications on the Cloud
• Many of these ISV solutions require GPU accelerated
graphics
• The Graphics Certification Program helps ISVs and their
customers achieve the maximum possible graphics
performance on AWS

39. Graphics Certification Program
• AWS partners with ISVs to:
• Test and optimize applications for AWS platform
• Invest in certifying AWS as a supported platform
• Set up customer support mechanics
• Invest in co-marketing
• Interested in having your application certified?
• Contact elastic-graphics-preview@amazon.com

40. Thank You
Sign up for Elastic GPUs Preview: https://pages.awscloud.com/elastic-gpus-preview.html
If you have any questions, please send us an email at elastic-graphics-preview@amazon.com