This document discusses high-performance parallel computing (HPC) solutions from ANSYS for computational fluid dynamics (CFD). It provides examples of how HPC allows companies to reduce turnaround time, examine more design variants faster, and simulate larger/more complex models. Specific applications from racing boat design, turbocharger optimization, and oil field processing are described. Performance benchmarks show scaling of ANSYS CFD solvers on multi-core CPUs and GPUs, with efficiencies over 80% on large core counts. HPC is presented as key to remaining competitive by removing computing limitations and allowing larger simulations.

1. © 2014 ANSYS, Inc. July 11, 20141
Customer Examples of High-Performance
Parallel Computing for CFD
Wim Slagter, Ph.D
ANSYS, Inc.
Series of 4 Presentations on HPC
1: High-Performance Parallel Computing for FEA
2: High-Performance Parallel Computing for CFD
3: High-Performance Parametric Computing for FEA
4: High-Performance Parametric Computing for CFD

2. © 2014 ANSYS, Inc. July 11, 20142
High Performance Computing (HPC) at ANSYS:
An ongoing effort designed to remove
computing limitations from engineers who
use computer aided engineering in all phases
of design, analysis, and testing.
It is a hardware and software initiative!
HPC Defined

3. © 2014 ANSYS, Inc. July 11, 20143
Why HPC?
Impact product design
Enable large models
Allow parametric studies
Turbulence
Combustion
Particle Tracking
Assemblies
CAD-to-mesh
Capture fidelity
Multiple design ideas
Optimize the design
Ensure product integrity

4. © 2014 ANSYS, Inc. July 11, 20144
Why HPC?
Results indicate very substantial returns for investments in HPC:
 $356 dollars on average in revenue per dollar of HPC invested
 $38 dollars on average of profits (or cost savings) per dollar of HPC invested
Key Findings: Financial ROI Results
Source: IDC report “Creating Economic Models Showing the Relationship Between Investments
in HPC and the Resulting Financial ROI and Innovation”; October 2013, IDC #243296, Volume: 1.

5. © 2014 ANSYS, Inc. July 11, 20145
Take Advantage of the HPC Revolution
Recent advances have revolutionized the
computational speed available on the desktop
• Multi-core processors
o Every core is really an independent processor
• Large amounts of RAM
• SSDs
• GPUs

6. © 2014 ANSYS, Inc. July 11, 20146
Typical HPC Growth Path
Cluster UsersDesktop User
Workstation and/or
Server Users

7. © 2014 ANSYS, Inc. July 11, 20147
Our HPC Solutions for CFD
HPC-enabled solutions:
• Fluid Dynamics (bundled and solver) products:
o Fluent, CFX, CFD-Flo, CFD Professional, Icepak, Polyflow
HPC processing solutions for running:
• A simulation – in parallel (simultaneously) – on multiple cores:
o ANSYS HPC, ANSYS HPC Pack, ANSYS HPC Workgroup
• Parametric simulations on multiple cores simultaneously:
o ANSYS HPC Parametric Pack

8. © 2014 ANSYS, Inc. July 11, 20148
High-Performance Parallel Computing

9. © 2014 ANSYS, Inc. July 11, 20149
Summary
Design Impact
HPC Parallel
Usingtoday’smulticorecomputersiskeyfor companiesto
remaincompetitive.ANSYSHPCproductsuiteallowsscalability
towhatevercomputationallevelrequired,fromsingle-useror
smallusergroupoptionsat entry-leveluptovirtuallyunlimited
parallelcapacityorlarge usergroupoptionsat enterpriselevel.
• Reduce turnaround time
• Examine more design variants faster
• Simulate larger or more complex models

10. © 2014 ANSYS, Inc. July 11, 201410
Application Example
Benefit of HPC Parallel
- Examine More Design Variants
Objective
Advance in racing boat design to sustain medal-
winning performances at Olympic games.
ANSYS Solution
• ANSYS CFX is used to optimize the fluid
dynamics for different classes of racing kayaks.
• Using HPC, transient simulations of moving
boats can be accomplished in just two or three
days.
Design Impact
Using HPC, the FES engineers were able to
efficiently consider up to 20 different virtual
designs per boat class, and from those 20 designs
they gained enough confidence to build a single
prototype for testing.
Images courtesy of FES Click here to read the full story

11. © 2014 ANSYS, Inc. July 11, 201411
Application Example
Benefit of HPC Parallel
- Increase High-Fidelity Insight
Objective
Full-stage simulations of turbochargers for diesel engines
are needed to reliably understand and optimize their
performance prior to physical prototyping.
ANSYS Solution
• ANSYS CFX simulations deliver near-linear parallel
processing on 160-core HPC system upgrade.
• ANSYS HPC performance delivers ability to consider 5
full-stage compressor or turbine designs in a few hours
(compared to many days prior to upgrade).
Design Impact
ANSYS HPC is enabling Cummins to use larger models
with greater geometric details and more-realistic
treatment of physical phenomena to generate results in
less time.
Images courtesy of Cummins Turbo Technologies Click here to read the full story

12. © 2014 ANSYS, Inc. July 11, 201412
Application Example
Benefit of HPC Parallel
- Increase High-Fidelity Insight
Objective
Overtake the technological challenges on flow assurance and
subsea oil processing present on the new pre-salt oil fields.
ANSYS Solution
• Transient multiphase simulations with ANSYS Fluent are
used to understand the sand transportation inside the
kilometres long production lines
• ANSYS HPC performance together with advanced
multiphase models and dynamic meshing features enable
Petrobras to virtually reproduce critical scenarios and
complex operation.
Design Impact
Very detailed CFD simulations are providing Petrobras
important physical insights that are guiding the design of the
new tendency of upstream processing systems at oil industry.

13. © 2014 ANSYS, Inc. July 11, 201413 Intel Xeon E5-2690v2 processors (3 GHz, 20 cores total)
with 128 GB of RAM.
0
200
400
600
800
1000
1200
1400
1p 2p 4p 6p 8p 10p 12p 14p 16p 18p 20p
solver
ratings
Processes (or Cores)
Geometric mean
0
2
4
6
8
10
12
14
16
1p 2p 4p 6p 8p 10p 12p 14p 16p 18p 20p
Speedup
processesSpeedup
Higher
is
Better
HPC Parallel Scaling of ANSYS Fluent
- Faster with More Compute Cores on Dual Processors

14. © 2014 ANSYS, Inc. July 11, 201414 Each node has 2 X 12-core Intel Xeon E5-2600 v2 processors
(2.4 GHz, 1600 MHz) with 64 GB of RAM. InfiniBand FDR.
0
2
4
6
8
10
12
14
HPC Parallel Scaling of ANSYS Fluent
- Faster with More Compute Cores at Multiple Nodes
24 48 96 192 384
Number of Cores
Speedup

15. © 2014 ANSYS, Inc. July 11, 201415
 Hexa mesh (830.000 cells)
 Standard K-Epsilon
Turbulence Model
 VOF multiphase model
(3 phases):
 Molten Steel
 Foamy Slag
 Oxygen
0
1
2
3
4
5
6
7
12 24 36 48 60 72
Speedup
cores
ideal speedup
measured speedup
cores
overall
time (h)
measured
speedup
ideal
speedup
12 0.56 1.00 1
24 0.29 1.94 2
36 0.21 2.60 3
48 0.17 3.33 4
72 0.12 4.76 6
Courtesy of MORE S.r.l.
HPC Parallel Scaling of ANSYS Fluent
- Faster with More Compute Cores for Complex Physics

16. © 2014 ANSYS, Inc. July 11, 201416
R&D effort to improve HPC scaling in CFX
• Basic & physics specific scaling areas
• Significantly improved scalability
– Up to 89% efficiency at 2048 cores
– HPC improvements are “beta” level for R15.0
4X faster
Courtesy Siemens AG, Müllheim, Germany, Paper GT2013-94639
5X faster
• Six Stage Axial Compressor
• 13M nodes
• 14 domains, 12 mixing planes
• Duct case
• 150M nodes
HPC Parallel Scaling of ANSYS CFX
- Faster with More Compute Cores at Multiple Nodes

17. © 2014 ANSYS, Inc. July 11, 201417
Sedan Model
Sedan geometry
3.6M mixed cells
Steady, turbulent
External aerodynamics
Coupled PBNS, DP
Intel Xeon E5-2680;
8 CPU cores
2 Tesla K40 GPUs
CPU + GPU
Segregated solver
1.9x
Coupled solver
CPU only CPU only
15
Jobs/day
12
Jobs/day
27
Jobs/day
Convergence criteria: 10e-03 for all variables; No of iterations until convergence: segregated CPU-2798
iterations (7070 secs); coupled CPU-967 iterations (5900 secs); coupled 985 iterations (3150 secs)
NOTE: Times
for total solution
until convergence
HPC Parallel Scaling of ANSYS Fluent
- Performance on Newest GPUs at R15.0
Higher
is
Better

18. © 2014 ANSYS, Inc. July 11, 201418
16
Jobs/day
25
Jobs/day
HPC Parallel Scaling of ANSYS Fluent
- Performance on Newest GPUs at R15.0
CPU
Benefit
100%
25%
100%
56%
GPU
Cost
Simulation
productivity from
CPU-only system
Additional
productivity from
GPUs
CPU-only
solution cost
Additional
cost of adding
GPUs
Simulation productivity
(with an HPC Workgroup 64 license)
64 CPU cores
56 CPU cores +
8 Tesla K40
Truck Model
External aerodynamics
14 million cells
Steady, k-e turbulence
Coupled PBNS, DP
Intel Xeon E5-2680; 64
CPU cores on 8 sockets
8 Tesla K40 GPUs
Higher
is
Better

19. © 2014 ANSYS, Inc. July 11, 201419
ANSYS Advantages
HPC Parallel
- Final Remarks
• Superior and proven parallel scalability
above 80% efficiency with as low as
10,000 cells per CPU core, providing the
ability to
– Run bigger models at smaller hardware
– Run smaller models at higher core counts
• Solvers required for complex physics
(chemistry, multiphase) are highly
optimized to run fast and deliver
outstanding parallel scaling on today’s
multicore processors
• ANSYS provides flexible, scalable, and
cost-attractive HPC licensing!
Images courtesy of Voith Turbo and MORE S.r.l. Click here why scalability matters

20. © 2014 ANSYS, Inc. July 11, 201420
Watch recorded webinars by clicking below:
• Understanding Hardware Selection for ANSYS 15.0
• How to Speed Up ANSYS 15.0 with GPUs
• Intel Technologies Enabling Faster, More Effective Simulation
• Why HPC for ANSYS Mechanical and CFD
Click on webinars related to HPC/IT for more and upcoming ones!
Connect with me on LinkedIn by clicking on Wim Slagter
Further Information

21. © 2014 ANSYS, Inc. July 11, 201421
THANK YOU!