“Large Eddy Simulations of Ethanol Spray Combustion”, presented at http://3whpc.lcca.usp.br

1. Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Large Eddy Simulations of
Ethanol Spray Combustion
Dr.-Ing. Flavio Cesar Cunha Galeazzo, flavio.galeazzo@usp.br
Prof. Dr.-Ing. Guenther Carlos Krieger Filho
3. Workshop de High Perfomance Computing 17. April 2015

2. Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Experimental setup
 PhD of Newton K. Fukumasu (2010 - 2014)
 Open burner derived from an automotive fuel injector
and a swirler
 Laser diagnostics (PDI, LIF e PIV)
3WHPC-LCCA Flavio Galeazzo 2

3. Laboratory of Thermal and Environmental Engineering
Combustion Research Center
100 Hz 250 Hz 400 Hz
Case Injection
frequency
Duty
cycle
Combustion pattern
ESF100 100 Hz 29% Long anchored flame
ESF250 250 Hz 43% Short anchored flame
ESF400 400 Hz 59% Lifted flame
Boundary conditions
Air flow 20 m3 / h
Ethanol flow 2 L / h
3WHPC-LCCA Flavio Galeazzo 3

4. Numerical setup
 OpenFOAM 2.1.1
 LES (Large Eddy Simulation)
 Smagorinsky subgrid scale model (SSG)
 Spray
 Lagrangian particle tracking
 Combustion model
 Simple “mixed is burned” approach
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
3WHPC-LCCA Flavio Galeazzo 4

5. Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Grid 1 Grid 2
Number of elements 600 000 1.6 million
Grid sensitivity
 Hexahedral elements
 Two grids with different
refinement
 Cold flow
 Better agreement with
refined grid
 Grid 2 was used for the
LES simulations
3WHPC-LCCA Flavio Galeazzo 5

6. • Cold flow
• Mean axial velocity UX at the
centerline
• Experiments
• RANS simulation using
Reynolds-stress turbulence
model
• LES using Smagorinky SSG
• The negative velocities indicate
the recirculation zone created by
the swirl flow
• Much better agreement with LES
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
3WHPC-LCCA Flavio Galeazzo 6

7. • Combustion
• Axial velocity(m/s)
• Temperature (K)
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
7

8. 100 Hz 250 Hz 400 Hz
• Experiment
chemiluminescence
• Mean temperature (K)
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
8

9. Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Computational resources
 BlueGene/P – USP/Rice
 42 racks, 128 CPUs, 512
computing cores each
 4 GB per CPU
 Proprietary interconnect
 SGI Altix cluster – LETE
 12 nodes, dual CPU, 8
computing cores each
 24 GB per node / 12 GB per
CPU
 Infiniband interconnect
Problems using BG/P
 Compilation of OpenFOAM in
the BG/P
 Need GCC compiler – can’t use
native IBM XL compiler
 Need shared libraries – complicated
to setup in the BG/P
 Run the OpenFOAM
simulation in the BG/P
 Need aprox. 2.5 GB per grid partition
 Can’t use all computing cores of
each CPU
3WHPC-LCCA Flavio Galeazzo 9

10. Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Performance comparison
 BlueGene/P – USP/Rice
 42 racks, 128 CPUs, 512
computing cores each
 4 GB per CPU
 Proprietary interconnect
 SGI Altix cluster – LETE
 12 nodes, dual CPU, 8
computing cores each
 24 GB per node / 12 GB per
CPU
 Infiniband interconnect
SwirlFlameLETE/1/H1/les_onlyFlow – cold flow
Grid = 1.6 million elements
Mode
Number of
partitions
Simulation
time (s)
Computing
time (s)
Relative
time
Blue
Gene/P
SMP 128 0,002 517 1
LETE openmpi 24 0,002 450 0,87
SwirlFlameLETE/1/H1/EPS250/les_fireFoam - combustion
Grid = 1.6 million elements
Mode
Number of
partitions
Simulation
time (s)
Computing
time (s)
Relative
time
Blue
Gene/P
SMP 128 0,002 8423 1
Blue
Gene/P
SMP 256 0,002 6077 0,72
Blue
Gene/P
SMP 384 0,002 5892 0,70
3WHPC-LCCA Flavio Galeazzo 10

11. Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Conclusions
 Better agreement of LES results with the experimental data
 More refined computational grids and longer simulation time are
required for LES
 For more refined grids there is a need of more computational power
 OpenFOAM 2.1 was successfully compiled in the BlueGene/P
 The performance is not ideal
 Old GCC compiler (version 4.3.2, from 2008)
 Memory usage of OpenFOAM do not permit the use of all computing cores
 Use of OpenMP is an alternative
 Current development, up to now no speedup using Paralution OpenMP solver
11

12. Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Thank you
Dr.-Ing. Flavio Cesar Cunha Galeazzo, flavio.galeazzo@usp.br
Prof. Dr.-Ing. Guenther Carlos Krieger Filho
12