HPC on Azure for Reserach

High Performance Computing (HPC)
Running compute-intensive workloads in Azure
Silvia Spera
Azure Infrastructure Technical Specialist
CRUI
Roma, 1 marzo 2019

© Microsoft Corporation
The challenge of
modern computing
Your technical and
business needs are
dynamic,
but your compute
environment is
static.

Research is the Lifeblood of
Scientific, Engineering, Health, Social, Technological
Breakthroughs

Factory
Layout
Clinical Trial
Simulation
Rocket
Design
Self-driving
Cars Hard disk design
Materials
Science
Fundamental
Science
Architecture
&
Engineering
Entertainment
Genomics
Drug Design
Environment
Impact
Computational
Chemistry
Quantum
computing
Cancer
Research
Circuit
Design
Risk
Management
Diagnostics
Crash Testing
Machine Learning
Product Design &
Safety Security/
Encryption
Logistics
Data
Science

Demand for HPC infrastructure
On-premises
On-premises HPC

On-premises HPC
On-premises
Challenges with
Data analyticsAI IoT
New business demands
Regulations
Keeping up with business demands
Maximizing your investment
Siloed environments limiting collaboration

On-premises
The HPC Cloud
Opportunity
Variable
demand
Fixed
demand
Expand your HPC
environment to the Cloud
The elasticity of the
cloud provides the
perfect accommodation
for variable demands of
compute resources.

The HPC Cloud
Opportunity
All
demand
Build your HPC
environment in the Cloud
A fully native, cloud-
based HPC environment
ensures optimized
business agility by
accommodating an on-
demand usage model
for compute resources

Existing
apps
Clone
to cloud
Start using cloud
without rewriting
applications
Hybrid
workflows
Simple to
optimize
infrastructure
Cloud-native
apps
Create new
services
and modernize
apps that matter
Azure for every HPC pattern
End-to-end
workflows in
the cloud
Cloud
workflows

Azure value for High Performance Computing
Most performant
infrastructure
Granular cost control
and governance
Open and
integrated

Productive TrustedHybrid
Azure HPC Value Pillars

Most performant infrastructure
Specialized infrastructure
for Super Computing
“The bare performance figures [of Azure] are best
of all cases studied, even when compared with the
traditional supercomputing system of reference.”
HPC Benchmarking Study, Feb 2017, Exabyte Inc.
Virtual Machines, Cray in Azure
Performant hybrid
storage with Azure
Easy management with Batch and Azure CycleCloud

Specialized infrastructure for HPC and AI
Cray in Azure
IB Connected CPU/GPU/Storage available in cloud
High-performance VMs
Tightly coupled parallel jobs
GPU-enabled VMs
NV—Graphic-based applications
NC—Advanced simulation
ND—Artificial Intelligence (Deep Learning)
H N
Compute-optimized VMs
Batch processing, Monte Carlo simulations
F
Large memory VMs
Large databases
M
FPGA
PGA Microservices— AI/Edge
>80,000 IOPs - Premium Storage
Low latency, high throughput apps
RDMA over …. For multi-node scaling

HPC Resources
on Azure
Azure HPC VMs
Azure GPUs
Bare Metal

Specialized Compute Fleet
Virtual Machines – HPC
FPGA Microservices –
AI/Edge
IB Connected
CPU/GPU/Storage
available in cloud
NC2 – Advanced Sim (P100-X)
ND1 – AI Inferencing (P40)
ND2* – AI Training (V100/V100
SXM)
H N
A D F G
L

No-compromise CPU and GPU based resources
• Up to 16 cores, 3.2 GHz E5-2667 V3
Haswell processor
• Up to 224 GB DDR4 memory, 14GB per
core
• FDR InfiniBand @ 56 Gbps, 2.6
microsecond latency
• 2 TB of local SSD
• Up to 4 NVIDIA Tesla K80 GPUs
• Up to 24 cores
• Up to 224 GiB memory
• Up to 1440 GiB of local SSD
• FDR InfiniBand
• Up to 4 NVIDIA Tesla M60 GPUs
• Up to 24 cores
• Up to 1440 GiB of local SSD
• Up to 4 NVIDIA Pascal P40 GPUs
• Up to 24 cores
• Up to 3 TB of local SSD
• FDR InfiniBand
• Up to 4 NVIDIA Pascal P100 GPUs
• Up to 24 cores
• FDR InfiniBand
• Up to 4 NVIDIA Volta V100 GPUs
• Up to 24 cores
• FDR InfiniBand
HPC VMs on Azure
• Up to 44 cores, Intel Xeon Platinum
processor
core
• EDR InfiniBand @ 100 Gbps
• 700 GB NVMe
• Up to 60 cores, AMD EPYC processor
core
• EDR InfiniBand @ 100 Gbps
• 700 GB NVMe
• Up to 4 NVIDIA Tesla M60 GPUs
• Up to 24 cores
• Up to 2,948 GiB of local SSD
H-Series:
Most powerful
CPU virtual
machines with
optional RDMA
N-Series:
GPU virtual machines
specialized for
graphic-intensive
workloads

VM Scale Sets
Deploy & manage groups of identical
VMs at very large scale
Scale multiple VM sets utilizing
different VM types
Automatically scale by policy
without pre-provisioning
Enjoy steep discounting with low
priority VMs
Simple ways to scale up your environment

Azure GPUs

Rendering Deep-Learning/AIHPC/SimulationVisualization
Oil & Gas and SciencesHealthcare & ResearchMedia, Entertainment &
Gaming
AerospaceAutomotive
Financial Services
RetailManufacturing

GPUs enable End to End Workflows in the Cloud
Storage & Shared Workspace
CONSUME
WorkstationsCompute
Design & Create
CREATE
VM
GPU
Satellite officeDesign review
Post-Processing
3D Print / Prototype Main office
On the road
Simulate & Render

• Excellent for accelerating machine learning and HPC workloads
• Volta SXM GPU instances – 8X NVIDIA V100 GPUs interconnected with NVLink mesh
• Tensor Core technology to deliver over 100 TFLOPS of deep learning performance
• Skylake based processor with premium storage support (SSD backed)
• Specs:
• 640 NVIDIA Tensor Core
• FP64 - 7.8 TFLOPS of double precision floating point performance
• FP32 – 15.7 TFLOPS of single precision performance
• GPU Memory 16 GB
• 300 GB/s GPU interconnect through NVLink
ND v2 – Volta Generation GPU Compute
ND40s_v3
Cores 40 cores
GPU 8 x V100 SXM
Memory 768 GB
Local Disk ~1.3 TB SSD
Network Azure Network + NVLink GPU interconnect
http://aka.ms/AzureNDv2GPUs

• Get faster results for the your graphic intensive
2D and 3D applications with visualization
optimized GPU instances featuring NVIDIA Tesla
M60 GPUs
• Broadwell based CPU processor with doubled
memory from previous generation (up to 448 GB)
• Premium storage support (SSD backed)
• Grid license included with each GPU instance
• Specs:
• 2048 NVIDIA CUDA cores per GPU
• 36 H.264 1080p30 streams
• GPU Memory 8 GB/GPU
NV v2 – Updated GPU Visualization Platform
NV6s_v2 NV12s_v2 NV24s_v2
Cores 6 12 24
GPU 1 x M60 2 x M60 4 x M60
Memory 112 GB 224 GB 448 GB
Local Disk ~700 GB SSD ~1.4 TB SSD ~3 TB SSD
Network Azure Network Azure Network Azure Network
GRID Licenses 1 2 4
http://aka.ms/AzureNVv2GPUs

GPU Accelerated Compute Family
NC NC v2 NC v3
Cores 6, 12, 24 6, 12, 24 6, 12, 24
GPU 1, 2, or 4 K80 GPU 1, 2, or 4 P100 GPU 1, 2, or 4 V100 GPU
Memory 56/112/224 GB 112/224/448 GB 112/224/448 GB
Local Disk ~380/~680/~1.5 TB SSD ~700/~1.4/~3 TB SSD ~700/~1.4/~3 TB SSD
Network Azure Network + InfiniBand (largest size only)
GPU Accelerated Deep Learning Family GPU Visualization Family
ND v1 ND v2 NV v1 Nv v2
Cores 6, 12, 24 40 6, 12, 24 6, 12, 24
GPU 1, 2, or 4 P40 GPU 8 V100 SXM GPU 1, 2, or 4 M60 GPU 1, 2, or 4 M60 GPU
Memory 112/224/448 GB 768 GB 56/112/224 GB 112/224/448 GB
Local Disk ~700/~1.4/~3 TB SSD ~1.3 TB SSD ~380/~680/~1.5 TB SSD ~700/~1.4/~3 TB SSD
Network
Azure Network + InfiniBand
(largest size only)
Azure Network + NVLink
GPU interconnect
Azure Network Azure Network

Bare Metal options
on Azure

A dedicated supercomputer on
your virtual network
Innovate faster to solve your
toughest challenges
Access to advanced cloud
solutions and AI
Bring your simulation data and
services closer together
Simplify the management of
your infrastructure
Cray in Azure

Your Cray supercomputer running in
Azure, close to your Azure services
Rely on a dedicated, built-
to-spec Cray XC or SC
supercomputer for your most
demanding workloads
Connect to the broad range
of Azure services on your
Azure Virtual Network
Access the Cray as a managed
service in the cloud as OpEx
instead of maintaining a
specialized infrastructure with
high up-front costs
Cray in Azure

Microsoft Azure Data Center
Customer’s Virtual Network
OFFERINGOFFERING
Dedicated XC or CS series with attached ClusterStor
storage systems customized for each customer and
installed in Microsoft owned data center, on the
customer’s virtual network. Allows for capacity
expansion without facilities management. Long term
commitment required.
Seamless connectivity to Azure Data Center network
and all Azure services
Cray provides sys admin, monitoring, maintenance &
support via Cray’s (or Customer’s) Virtual Network on
Azure
Microsoft provide hosting, L1 support and Azure Data
Center network to Azure services
Customers access and run M&S and AI & Advanced
Analytics workloads on XC’s an ExpressRoute
connection
Azure Services
High
Speed
Network
Microsoft
Cray
Customer
Data LakeVM’s
IOT Hub
AI & ML
Azure Support
Cray Support
Storage
CycleActive
Directory
App Services

Virtual Compute Farm
Avere Virtual FXT
Bucket 1 Bucket 2
Bucket nAzure Blob
Avere
Physical
FXT
NAS Object
Customer needs Azure delivers
Low-latency file access Edge-core architecture
Scalable performance and HA
Scale-out clustering
(3 to 24 nodes per cluster)
Familiar NFS and SMB interfaces FlashCloud file system for object storage
Manage as a single pool of storage Global namespace (GNS), FlashMove
Data protection Cloud snapshots, FlashMirror
High security
AES-256 encryption
(FIPS 140-2 compliant), KMIP
Efficiency LZ4 compression
Performant hybrid storage with Azure

Microsoft Avere vFXT
Faster, more accessible data for Edge HPC
High-performance file access for HPC applications
Performance. Automatic identification and caching of hot data needed for
demanding workloads, minimizing storage latency for heavy-read
environments.
Access. File-based applications can access Azure compute resources from either
on-premises network-attached storage (NFS/SMB) or Azure Blob (REST)
Flexibility. Get the flexibility of having the scale of Azure at your fingertips to
support critical workloads
Azure Marketplace
Now in Preview
Media & Entertainment | Life Sciences | Financial Services | Manufacturing | Oil & Gas | Government

Building a Hybrid Cloud – Avere vFXT for Azure
Virtual Network
Subnet
Your Azure Storage
Account
Mountpoint
Address
Range
HPC Cluster
Azure Blob
Container
Avere vFXT
Cluster
NFS 3
Storage
Endpoint
Azure
subscription
Mountpoint
Aliases
(RRDNS)
Gateway Subnet
Your Data Center
ExpressRoute
/export1
Network Attached
Storage

Fluent 280M Cell Open
Race Car Model
Benchmark Scaling on
Microsoft Azure
WallTime(s)
Scale(X)
2000
1800
1600
1400
1200
1000
800
600
400
200
0
9.000
8.000
7.000
6.000
5.000
4.000
3.000
2.000
1.000
0.000
128 256 384 512 640 768 896 1024
Total Wall Time Scale Theoretical
Cores

Azure CycleCloud
running clusters
Azure Batch
running jobs
Azure clusterHybrid/burstHPC as a service
On-premises
Client
Client App or Web
portal
All HPC resources in the cloud
Azure
Batch
Resourcepool
Client
Computenodes
Head node Compute nodes
Client
Head node
Compute nodes
On-premises All HPC resources in the cloud
Services for workload management

Azure Batch
API for creating pools of resources and the schedule jobs and tasks on them
PaaS
Cloud Services
IaaS
VM / VMSS
Hardware
Get and manage VMs
Start the tasks
Queue tasksMove task input and output
Install task applications
Scale up and downTask failure? Task frozen?
Manage and
authenticate users
Azure Batch
VM management and job scheduling

Access via APIs, CLIs, and UIs
• .NET, Java, Node.js, Python, REST
• PowerShell, x-plat Azure CLI
• Azure Portal, Batch Labs x-plat client UI
Choice of VMs
• Windows or Linux. Choose the VM
size most suited to your application
• Standard or custom images
• Automatically scale the number of
VMs to maximize utilization
• Windows pool can use AHUB
• Use low-priority VMs
Rich app management
• Get apps from blobs, Batch app
packages, package managers, custom
VM images
• Docker container images
Job scheduling
• Supports both embarrassingly parallel and
tightly-coupled MPI jobs
• Run > 1 task in parallel per node
• Detect and retry failed tasks
• Can set max execution time for jobs and tasks
• Task dependencies
• Job prep and cleanup tasks
Monitoring
• VM monitoring and auto-recover
• Metrics and logs available via Portal and API
Azure Batch Capabilities

Azure CycleCloud
runs compute clusters
IT management
Link workflows for internal and
external clouds
Use Active Directory for
authentication and authorization
Provide secure and consistent access
User empowerment
Able to cloud-enable existing workflows
Enable instant access to resources
Provide auto-scaling, error handling
Business management
Able to link usage to spend
Provide tools to manage and control costs

Hybrid/Clustered Big Compute Lifecycle
Optimization
Provisioning
Cluster
Configuration Monitoring
Internal
Admin
Scope Configure
Run on
Cloud OptimizeUser

On-Premise Cluster
for simulation and
predictive analyses
Gap #1: Hybrid / Migration
Using templates to mirror the on-premise environment,
CycleCloud provisions an identical environment into
which the application can be loaded without rewriting
User Apps
& Scripts
Filling the gaps in moving a strategic HPC application environment to Azure
H
Nc
D, F
vNet
Avere vFXT
Engineer, Quant,
or Data Scientist
Line of Business Manager
Gap #2: Governance
CycleCloud solves governance issues by providing a
policy-based system for access, authorizations, cost
controls, and compliance audit reporting.
IT/Admin
Azure CycleCloud Scenario:

Making Users and IT Successful
LOB &
User
Inputs
IT LOB
Inputs
Internal

•
•
•
•
•
When to Use Azure Batch or CycleCloud
Azure Batch – Running jobsCycleCloud – Running clusters

Open and integrated
Robust
partner
ecosystem
Microsoft and
open source
software support

Robust partner ecosystem

Microsoft and open source software support
.NET

Optimize cloud
efficiency
Monitor cloud
spend
Drive organizational
accountability
With Cost Management, you can:
Across your multi-cloud environment

Cost savings
Ensures Reserved Instances
for persistent infrastructure
Offers flexible consumption and
cost savings with low-priority VMs
Provides per-second
billing for VMs
Azure services

The largest compliance portfolio in the industry

ENERGY & UTILITIES
Seismic Processing
Reservoir Simulation
Digital Oil Fields
Smart Grids
Market Use-Cases and Opportunities
CYBER SECURITY
Insider Threat Detection
Continuous Monitoring
Data Loss Monitoring and
Analysis
GOV’T AND DEFENSE
National security
High energy physics
Civil engineering
Material science
FINANCIAL SERVICES
Risk and Stress Testing
Strategy Back Testing
Anti Money Laundering
KYC – Know Thy
Customer
EARTH SCIENCES
Weather Prediction
Climate change
Oceanography
Air Quality
HEALTHCARE AND
LIFE SCIENCES
Personalized medicine
Genomic sequencing
Drug Discovery
Early Disease Detection
through Advanced Imaging
MANUFACTURING
Air Flow Modeling
Crash Simulation
Predictive analytics
Autonomous vehicles
HIGHER EDUCATION
Astrophysics
Molecular Modeling
Computational chemistry
Neuro informatics
COMMUNICATIONS
AND MEDIA
Social media insights
Network intelligence
Image Classification
CONSUMER-DRIVEN
INDUSTRIES
Consumer Sentiment
Best Next Offer Analytics
Customer Churn
Supply Chain Optimization
Dynamic Pricing
Inventory Forecasting
Unleashed by data and advanced computing, AI and Machine Learning are impacting every industry

We are committed to the Research community
51
We want the cloud to work the way the
academic research community works.
That’s why we’re waiving the Internet
egress fee for qualified customers.
• Ensures predictability and stability
in cloud costs
• Removes a significant barrier to
cloud adoption, allowing researchers
to move data freely
• Paves the way for researchers to
accelerate the pace of the important
work they’re doing

Researchers in Sao Paolo,
Brazil study how carbon
dioxide, water, nitrogen, and
other nutrients cycle through
plants, animals, and
microorganisms in this
complex ecosystem using 700
sensors in 15 forest plots
https://www.microsoft.com/en-us/research/blog/cloud-
computing-helps-make-sense-of-cloud-forests/

Researchers at University of
Texas used predictive
modelling to calculate flood
predictions to support the
efforts of the first responders
at risk of potential flooding.
https://www.microsoft.com/en-us/research/blog/coping-
with-floodsof-water-and-data/

Researchers in the particle
physics group at the
University of Victoria created
a flexible cloud system for
large workloads such as high-
energy physics computing.
The system also streamlined
user workflow, sparking
collaboration in the research
community.
http://customers.microsoft.com/en-us/story/university-
victoria-azure-cloud-services-canada-education

Financial services Workloads
Key partnerships

Financial services workloads
Monte Carlo simulations
Risk analytics
CCAR
FRTB
Regulatory reporting
Scripts at scale
Batch processing
Code analysis
Unit testing
Regression testing

Key financial services partners

Achieved consistent experience in Azure
on-premises with familiar tools and processes
Gained cost savings of up to 55 percent and
year-over-year infrastructure savings by scaling
down unused compute
Improved customer service with quicker data
processing and faster results

Automotive
industry
Workloads
Key partnerships

Azure HPC for the automotive industry
Finite Element Analysis (FEA)
Computational Fluid
Dynamics (CFD)
Multibody Dynamics (MBD)
Simulation Program with
Integrated Circuit Emphasis
(SPICE) analysis
Optimization
HPC/Simulation
Concept vehicle styling
Digital concept vehicle
Clay model replacement
Marketing campaign
Web, media, print
Engineering
Windshield cockpit glare
Rendering
Cloud-based engineering
(cloud VDI)
Cloud-based post
visualization
of simulations
Visualization
Autonomous driving
Advanced Driver Assistance
Systems (ADAS)
Adaptive cruise control
Auto parking
Navigation systems
Collision avoidance/warning
Lane departure warning
Deep Learning/AI

Key automotive partners

Stay ahead of the competition by accessing a
powerful infrastructure for real-time race simulations
and deep analysis of track, driver and vehicle data
Ensure significant cost savings versus building
an on-premises HPC environment
Provide rapid decision making using Azure
Machine Learning and intelligent services

Oil and gas Workloads
Key partnerships

Oil and gas workloads
Process raw seismic data
Perform bulk upload of seismic data
Use in-house or commercial
applications to process seismic data
Seismic processing Reservoir simulations Remote Visualization and AI
Accelerate onboarding
and knowledge transfer
Increase collaboration and productivity
of field workers
Create more accurate reservoir models,
optimize drilling, and identify risks
Evaluate production data
to update models
Optimize equipment and enable
predictive maintenance
Help provide development of
new oil fields
Ensure that simulation helps predict
oil production and productivity
Use commercial tools like Nexus or
Intersect to run Reservoir simulation
in Azure

Key oil and gas partnerships

Media and
entertainment
Workloads
Key partnerships

Media and entertainment workloads
Hybrid/burst for peak demand
Large-scale cloud-native workflows with Batch
Rendering Remote visualization
Shared workstations for collaboration

Key media and entertainment partners

Azure value for
manufacturing
Workloads
Key partnerships

Manufacturing workloads
Finite Element Analysis (FEA)
Computational Fluid
Dynamics (CFD)
Multibody Dynamics (MBD)
Simulation Program with
Integrated Circuit Emphasis
(SPICE) analysis
Optimization
HPC/Simulation
Digital concept product
Physical models
Marketing campaign
Web, media, print
Engineering
Windshield cockpit glare
Rendering
Cloud-based engineering
(cloud VDI)
Cloud-based post visualization
of simulations
Visualization
Image recognition for
quality assurance
Deep Learning/AI

Key manufacturing partners

Summary

Only Microsoft has all the pieces to meet customers where
they are in their HPC cloud journey
Comprehensive
portfolio
Only cloud vendor to
offer IaaS, PaaS, hybrid
and supercomputer
solutions for HPC
Extensive partner
ecosystem
Unique programs &
industry leaders ready
to collaborate with
your business
Easy to
Transition
Tools for hybrid
infrastructure, cloud
clusters, data staging,
and cloud apps
Investment
In HPC
Dedicated HPC
engineering
and customer
engagement teams
Intelligent
Cloud
Join HPC with AI and
IOT for Digital Twins
and building next
generation models

Making HPC a reality
HPC challenges Azure solves these
Getting your workload into the cloud Simple, managed access to HPC
Supporting hybrid use cases
Azure CycleCloud and Avere vFXT for burst, including data
and executables
Moving data and apps Fast, repeatable, and scalable deployment
Managing bandwidth, security and users Cost, user, and access controls
Accessing the required technologies within the time given
Strategic investments in HPC-specific technologies & service
frameworks built into Azure
Building cloud-native applications Azure Batch for resource provisioning and job scheduling

Next steps
Learn more about Azure for HPC
https://azure.microsoft.com/solutions/high-performance-
computing/
Explore Azure solutions for your industry
https://azure.microsoft.com/solutions/
Engage a partner to help implement your high-
performance solution
https://azure.microsoft.com/partners/directory/?solution=hi
gh-performance-computing

Compute options for all types of apps

Intel Broadwell E5-2673 v4
Hyper-Threaded CPUs
Up to 64 vCPU’s, 256GB RAM
Intel® Xeon® Platinum 8168
processor (Skylake)
Intel Haswell E5-2673 v3
Lowest cost, flexible CPUs
Up to 8vCPUs, 32GB RAM
AMD EPYC™7551 processor
8x1.9 TB SSDs

Intel Xeon E5-2673 v4 (Broadwell)
Hyper-Threaded CPUs
Up to 64 vCPUs, 432GB RAM
Intel Xeon E7-8890 v3 (Haswell)
Largest VMs in Azure
Up to 128 vCPUs, 4TB RAM
Nvidia V100 SXM GPU
Infiniband networking
Deep learning, AI
‘
Nvidia M60
V100 PCIe
‘
Skylake Processor
‘

HPC
Academia
Automotive
Oil & Gas
Ship engineering
Banking
Insurance
Energy
Defense &
Aerospace
Pharmaceutical
Food &
beverages
Healthcare
Bio Science
Oceaneering
Weather forecasting
Chemical
engineering
Engineering &
construction
Graphics &
rendering
Fluid
dynamics
Structural
simulations
Crash
Simulations
Deep
Learning
Genomics
Molecular
Modelling
Risk
Simulations

Most performant HPC Virtual machines
Broadest GPU portfolio
Largest compliance portfolio of any cloud
Fastest Networking with InfiniBand
and RDMA support
Largest global footprint
Per second billing for VMs
Native integration into HPC client apps

Cost control and governance
Provides monitoring Includes cost-
saving features
Offers the largest global
footprint and compliance
portfolio of any cloud
Azure cost management

HPC on Azure for Reserach

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to HPC on Azure for Reserach

Similar to HPC on Azure for Reserach (20)

More from Jürgen Ambrosi

More from Jürgen Ambrosi (20)

Recently uploaded

Recently uploaded (20)

HPC on Azure for Reserach