This document discusses IBM's PowerAI platform for deep learning. It provides the following key points: 1. PowerAI is built on open source tools and IBM's POWER processors to provide an enterprise-ready software distribution for data scientists to develop deep learning models faster. 2. The Power9 CPU architecture features NVLink connections that provide faster GPU-CPU bandwidth compared to PCIe, allowing PowerAI to support larger and more complex deep learning models. 3. PowerAI with distributed deep learning capabilities can scale to train models on 256 GPUs, achieving a 58x speedup compared to a single system for image classification tasks.

1. PowerAI
Mai 2018
François Etcheverry
Spécialiste Technique Client – Cognitive Systems
francois.etcheverry@fr.ibm.com

2. 2
Founding Members

3. 3
© 2016 OpenPOWER Foundation
3
Accelerates Technology roadmap
2015 2016 2017
POWER8 POWER8 with NVLink POWER9
OpenPower
CAPI Interface
Enhanced
CAPI & NVLink
Connect-IB
FDR Infiniband
PCIe Gen3
ConnectX-4
EDR Infiniband
CAPI over PCIe Gen3
ConnectX-5
Next-Gen Infiniband
Enhanced CAPI over PCIe Gen4
Mellanox
Interconnect
IBM CPUs
NVIDIA GPUs Kepler
PCIe Gen3
Volta
Enhanced NVLink
Pascal
NVLink
& Systems
Xilinx FPGAs UltraScale
CAPI, PCIe Gen3
UltraScale +
Enhanced CAPI, PCIe Gen3
UltraScale +
CAPI 2.0, PCIe Gen4

4. 4
© 2016 OpenPOWER Foundation
4
Power9
Graphics Memory
System
Memory
Graphics Memory
NVLinkPCIeGPU
CPU
Graphics
Memory
System
Memory
16 GB/s
75 GB/s
75 GB/s
75GB/s
OpenPOWER Technology: 5.6x Faster CPU-GPU
Connection via NVLink
GPUs Bottlenecked by PCIe Bandwidth
From CPU-System Memory
NVLink Enables Fast Unified Memory Access
between CPU & GPU Memories
NVLink
NVLink
System
bottleneck
GPU
GPU

5. 5IBM Systems
POWERAI
enterprise-ready
software distribution
built on open source
tools for ease
of development
performance
faster training times
for data scientists
POWERAI Vision
Data Science
Experience

6. 6
PowerAI with Large Model Support|
Train larger, more complex models
Large Model Support
POWER
CPU
DDR4
GPU
NVLink
Graphics
Memory
Traditional Model Support
CPUDDR4
GPU
PCIe
Graphics
Memory
Limited memory on GPU forces trade-
off in model size / data resolution
Use system memory and GPU to
support more complex and higher
resolution data
• Leveraging NVLink and CPU-GPU memory coherence enables larger and more complex models
• Improves model accuracy with more images and higher resolution images

7. 7
PowerAI Rel. 5 with Distributed
Deep Learning tech. preview
Performance…
Faster Training
and Inferencing
Near Ideal Scaling to 256 GPUs and Beyond17 Days Down to 7 Hours:
58x Faster
1 System 64 Systems
17 Days
7 Hours
ResNet-101, ImageNet-22K, Caffe with PowerAI DDL, Running on Minsky (S822Lc) Power System (https://arxiv.org/abs/1708.02188)

8. 8
8
• TensorFlow 1.4 Performance on IBM POWER9 with Nvidia V100
• Single node vs distributed
Date of testing : 26 november (single node) & 2 December (Multi nodes)
https://developer.ibm.com/linuxonpower/perfcol/perfcol-mldl/

9. 9
faster training times
for data scientists
Performance…
Faster Training
and Inferencing
https://arxiv.org/abs/1803.06333
Régression Logistique

10. 10
IBM Spectrum Conductor & Deep Learning Impact
Trained Model
Raw Data
Power AI
Deep Learning Frameworks
(Tensorflow & IBM Caffe)
New Data
Monitor &
Advise
Train
Test
Instrumentation
Model Training Inference
Deploy in Production using
Trained Model
(Inference as a Service)
Data Preparation
Iterate
Distributed & Elastic Deep
Learning (Fabric)
Parallel Hyper-Parameter
Search & Optimization
Spectrum Conductor
Deep Learning Impact
Existing Spectrum
Conductor
Third Party/Open
source
Network
Models
Hyper-
Parameters
Deploy
Pre-Processing
Training
Dataset
Testing
Dataset
Session Scheduler
Security
Data Connector
Report/log management
Notebook Spark ELKPython
Service Management (ASC/K8s)
ContainerGPU and Acceleration
Multi-tenancy

11. 11
Merci de votre attention