PCクラスタコンソーシアムの開催するPCCC23（第23回PCクラスタシンポジウム）「HPC基盤技術と生成AI」会員展示として公開中のスライドです。 詳細は下記のWebサイトへお越しください。 ▽PCCC23 「HPC基盤技術と生成AI」 （2023年12月7日～8日） https://www.pccluster.org/ja/event/pccc23/

1. AMD EPYC™
Processors for AI
Nov 2023

2. 2 |
[Public]
Broad Industry Impact
Self-driving cars can
recognize signage,
pedestrians, and
other vehicles to be
avoided
Monitor quality of
manufactured
products from food
items to printed
circuit boards
Automate checkout
lines and use product
recommendation
engines to offer
suggestions, whether
online or in
the store
Detect anomalies
including fractures
and tumors.
Use the same models
in research to assess
in vitro cell growth
and proliferation
Natural language
processing can use
spoken requests and
recommendation
engines to help point
customers to
solutions
AI-powered anomaly
detection helps stop
credit-card fraud,
while computer vision
models watch for
suspicious documents
Automotive Manufacturing Retail
Financial
Services
Medical
Service
Automation
AI extends and enriches common business workloads and activities

3. 3 |
[Public]
AMD Propels theAI Lifecycle
TRAINING
The most data- and processing-intensive
part of the AI lifecycle. Significant
computing power is required, and servers
equipped with AMD Instinct™ accelerators
are designed to accelerate the process
INFERENCING
Once trained, AI requires comparatively
less processing power to process
incoming data and business records in
real time. Inferencing happens close to
the data and AMD EPYC™ processors
are ideal for inferencing.

4. 4 |
[Public]
AMD EPYC™ Processors:
Inference Performance
1.78x
Up
to
SERVERS BASED ON AMD EPYC™ 9654 CPUS
RECOGNIZE VEHICLES AT 1.78X THE RATE OF
INTEL® XEON® Platinum 8940H CPU BASED SERVERS
Phoronix used the OpenVINO benchmark using INT8-FP16 data types to compare multiple CPU types. They measured a whopping 78% speedup on vehicle detection,
and a 14% speedup on age-gender recognition comparing a 2P Intel Xeon 8490H processor-powered server to a 2P AMD EPYC 9654 processor-powered server with
ATX-512 on. See endnotes SP5-192, -193
6207
11029
0 2000 4000 6000 8000 10000 12000
2 x Intel Xeon Platinum 8940H
2 x AMD EPYC 9654
OpenVINO FP16-INT8
Vehicle Detection FPS
(Higher is Better)
103184
118104
0 20000 40000 60000 80000 100000 120000 140000
2 x Intel Xeon Platinum 8940H
2 x AMD EPYC 9654
OpenVINO FP16-INT8
Age Gender Recognition Faces per Second
(Higher is Better)

5. 5 |
[Public]
256 Threads for End-to-EndAI Boost
Results may vary due to factors including system configurations, software versions and BIOS settings. As of 6/13/2023, see endnotes: SP5-051.
• Comparison derived from TPCx-AI benchmark
covering 10 end-to-end use cases covering
training, serving and throughput
• 128C AMD EPYC 9754 delivers up to an aggregate
of ~2.2x the AI test cases per min. vs. 60C Intel
Xeon Platinum 8490H
Outstanding end-to-end AI throughput
performance on a wide variety of use cases
831
1841
120 total cores/
240 threads
4 instances / 30 vCPUs per
256 total cores/
512 threads
8 instances / 30 vCPU per
Xeon® Platinum 8490H AMD EPYC™ 9754
~2.2x
Running 2Pserverswith 128C4thGenAMDEPYC™ 9754vs.60C4thGenIntel® Xeon® Platinum8490H
End-to-end AI data science pipeline
aggregate AI
use cases/min

6. 6 |
[Public]
ROCm™ Platform Vitis™ AI Platform
CPU Stack
Unified Inferencing Model StreamlinesAdoption
The Unified Inference Frontend (blue) provides a uniform way to link your inferencing software with the acceleration capabilities of
EPYC™ CPUs, AMD Instinct™ accelerators, and Versal™ and Zynq™ adaptive SoCs
The CPU-specific software stack includes a robust set of tools that accelerate deep learning and inference workloads

7. 7 |
[Public]
AI Ecosystem Enablement
Model Optimization
• ResNet50, ResNet101,
ResNet152
• MobileNet-v1, MobileNet-v2
• Inception V3, Inception V4
• AlexNet, GoogleNet
• RNNs, LSTMs, GRUs
• BERT-Base, BERT-Large
• DLRM
• Wide and Deep
Key Models that use Optimized ZenDNN Primitives
Computer
Vision
Natural Language
Processing
Recommendation
Systems

8. 8 |
[Public]
AMD Solutions forAI
Workload-optimized engines enableAI efficiency
AI Accelerators
Server CPUs
FPGAs and
Adaptive SoCs

9. Thank You

11. 11 |
[Public]
End Notes
SP5-051: TPCx-AI SF3 derivative workload comparison based on AMD internal testing running multiple VM instances as of 6/13/2023. The aggregate end-to-end AI throughput test is derived from the TPCx-AI
benchmark and as such is not comparable to published TPCx-AI results, as the end-to-end AI throughput test results do not comply with the TPCx-AI Specification. Configurations: 2 x AMD EPYC 9754 on Titanite
(BIOS and Settings: AMI Core Ver. 5.25, Project Ver. RTI1000F and Default BIOS settings (SMT=on, Determinism=Auto, NPS=1)), 1.5TB (24) Dual-Rank DDR5-4800 64GB DIMMs, 1DPC, SK Hynix SHGP31-500GM
500GB NVMe, Ubuntu® 22.04 LTS (8-instances, 30 vCPUs/instance, 1841 AI test cases/min); 2 x AMD EPYC 9654 on Titanite (BIOS and Settings: AMI Core Ver. 5.25, Project Ver. RTI1000F and Default BIOS
settings (SMT=on, Determinism=Auto, NPS=1)), 1.5TB (24) Dual-Rank DDR5-4800 64GB DIMMs, 1DPC, Samsung SSD 983 DCT 960GB, Ubuntu 22.04.1 LTS (6-instance, 28 vCPUs/instance, 1554 AI test cases/min);
2 x Intel(R) Xeon(R) Platinum 8490H on Dell PowerEdge R760 (BIOS and Settings: ESE110Q-1.10 and Package C1E, Default BIOS settings (C State=Disabled, Hyper-Threading, Turbo boost= enabled (ALL)=Enabled,
SNC (Sub NUMA)=Disabled)), 2TB (32) Dual-Rank DDR5-4800 64GB DIMMs, 1DPC, Dell 1.7TB NVMe, Ubuntu 22.04.2 LTS (4-instance, 30 vCPUs/instance, 831 AI test cases/min). Results may vary due to factors
including system configurations, software versions and BIOS settings. TPC Benchmark is a trademark of the TPC.
SP5-192: OpenVINO 2022.2 FP16-INT8 Vehicle Detection FPS with AVX-512 on comparison based on Phoronix Test as of 18 Jan 2023. Configurations: 2P 96-core AMD EPYC™ 9654 (11029 FPS) powered server
versus 2P 60-core Intel® Xeon® Platinum 8940H (6207 FPS) for 1.78x the performance. https://www.phoronix.com/review/intel-sapphirerapids-avx512/7. Testing not independently verified by AMD. Scores will vary
based on system configuration and determinism mode used.
SP5-193: OpenVINO 2022.3 FP16-INT8 Age Gender Recognition Faces per Second comparison based on Phoronix Test as of 18 Jan 2023. Configurations: 2P 96-core AMD EPYC™ 9654 (118104 Faces per Second)
powered server versus 2P 60-core Intel® Xeon® Platinum 8940H (103184 Faces per Second) for 1.14x the performance. https://www.phoronix.com/review/intel-sapphirerapids-avx512/7. Testing not independently
verified by AMD. Scores will vary based on system configuration and determinism mode used.
Reference “AI Inferencing with AMD EPYC Processors” : https://www.amd.com/content/dam/amd/en/documents/solutions/ai/ai-inferencing-amd-epyc-processors-white-paper.pdf

12. 12 |
[Public]
DISCLAIMERS AND ATTRIBUTIONS
The information contained herein is for informational purposes only and is subject to change without notice. While every precaution has been taken in the preparation of this document, it may contain technical
inaccuracies, omissions and typographical errors, and AMD is under no obligation to update or otherwise correct this information. Advanced Micro Devices, Inc. makes no representations or warranties with respect to
the accuracy or completeness of the contents of this document, and assumes no liability of any kind, including the implied warranties of noninfringement, merchantability or fitness for particular purposes, with respect to
the operation or use of AMD hardware, software or other products described herein. No license, including implied or arising by estoppel, to any intellectual property rights is granted by this document. Terms and
limitations applicable to the purchase or use of AMD’s products are as set forth in a signed agreement between the parties or in AMD's Standard Terms and Conditions of Sale. GD-18
©2023 Advanced Micro Devices, Inc. all rights reserved. AMD, the AMD arrow, EPYC, and combinations thereof are trademarks of Advanced Micro Devices, Inc Intel, the Intel logo and Xeon are trademarks of Intel
Corporation or its subsidiaries. SPEC®, SPECrate® and SPEC CPU® are registered trademarks of the Standard Performance Evaluation Corporation. See www.spec.org for more information. Other product names
used in this publication are for identification purposes only and may be trademarks of their respective companies.