Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Masashi Imano, profile picture
Uploaded byMasashi Imano
PDF, PPTX737 views

SGI OpenFOAM Cloud Benchmark Part 3, OpenFOAM Study Meeting for beginner@Kansai, 11th Aug. 2012

This document summarizes the results of benchmarks performed on an SGI Altix ICE8200 cluster using OpenFOAM to simulate 2D and 3D lid-driven cavity flows. The benchmarks tested scaling performance of OpenFOAM using OpenMPI and SGI MPT for different problem sizes (mesh resolutions) and numbers of compute nodes. In general, OpenMPI showed better strong scaling than SGI MPT, although using two InfiniBand rails with SGI MPT provided some improvement over single rail.

Engineering◦Spiritual◦Technology◦
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Masashi Imano SGI OpenFOAM Cloud Benchmark Part 3 1
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Test cluster conditions • Hardware: SGI Altix ICE8200 • CPU: Intel Xeon X5365 3.00GHz 4cores x 2 • Node: 16 nodes • Memory: 16GB/node (No swap memory) • Interconnect: InfiniBand DDR • OS: SUSE LINUX ENTERPRISE SERVER 10.3 • MPI: ✓ SGI MPT (1.2.6) ✓ OpenMPI (ThirdParty package distributed with OpenFOAM) 2
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) • OpenFOAM versions: 2.1.x • MPI (interconnect: InfiniBand): ✓ OpenMPI (Ver. 1.5.3), mpirun options: --mca btl openib,self,sm ✓ SGI MPT (Ver.1.2.6) • Number of InfiniBand rails: 1, 2 (export MPI_IB_RAILS=1 or 2) • Benchmark problem ✓ Flow: Lid-driven 2D cavity(tutorials), 3D cavity ✓ Re: 100 (laminar) ✓ solver: icoFoam (piso) ✓ Linear solver: p(AMG), U(PBiCG) ✓ Mesh: • 2D (Δt=10-6 s, 100 steps): 1000 2, 2000 2, 4000 2 • 3D (Δt=10-5 s, 50 steps): 100 3, 200 3 x Ux = 1 m/s d = 0.1 m y Calculation conditions 3
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 75 150 225 300 1 2 4 8 16 113.3 155.8 204.4 102.3 54.5 52.3 71.2 135.9 101.9 54.8 16.3 29.1 61.2 89.4 53.5 NumberofCases/Day Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 2D cavity, mesh:10002 (1 Million) 4
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 0.2 0.4 0.6 0.8 1 1 2 4 8 16 Parallelefficiency Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 2D cavity, mesh:10002 (1 Million) 5
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 18 35 53 70 1 2 4 8 16 63.0 43.4 35.3 17.7 8.7 33.0 29.0 33.6 17.7 8.8 10.812.4 21.1 16.5 8.5 NumberofCases/Day Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 2D cavity, mesh:20002 (4 Millions) 6
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 0.2 0.4 0.6 0.8 1 1.2 1 2 4 8 16 Parallelefficiency Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 2D cavity, mesh:20002 (4 Millions) 7
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 5 10 15 20 2 4 8 16 20.0 9.8 6.1 3.4 12.9 8.9 6.1 3.5 5.75.15.2 3.3 NumberofCases/Day Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 2D cavity, mesh:40002 (16 Millions) 8
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 0.2 0.4 0.6 0.8 1 2 4 8 16 Parallelefficiency(base:2nodes) Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 2D cavity, mesh:40002 (16 Millions) 9
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 225 450 675 900 1 2 4 8 16 531.1 835.8 675.2 456.2 203.4 264.8 488.7 421.3420.3 203.2 197.1 403.4 434.4 389.5 194.1 NumberofCases/Day Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 3D cavity, mesh:1003 (1 Million) 10
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 0.2 0.4 0.6 0.8 1 1.2 1 2 4 8 16 Parallelefficiency Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 3D cavity, mesh:1003 (1 Million) 11
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 38 75 113 150 1 2 4 8 16 142.9 131.9 72.2 29.9 15.5 89.9 107.4 66.5 29.7 15.5 78.8 94.6 62.4 28.9 15.2 NumberofCases/Day Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 3D cavity, mesh:2003 (8 Millions) 12
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 0.2 0.4 0.6 0.8 1 1.2 1 2 4 8 16 Parallelefficiency Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 3D cavity, mesh:2003 (8 Millions) 13
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 15 30 45 60 4 8 16 58.5 38.1 11.0 42.4 34.8 10.6 38.1 32.7 10.3 NumberofCases/Day Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 3D cavity, mesh:3003 (27 Millions) 14
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Benchmark results 0 0.5 1 1.5 2 4 8 16 Parallelefficiency(base:4nodes) Number of Nodes OpenMPI SGI MPT(1 Rail) SGI MPT(2 Rails) 3D cavity, mesh:2003 (8 Millions) 15
Aug. 11th 2012 OpenFOAM Study Meeting for beginner @ Kansai Masashi Imano (OCAEL Co. Ltd.) Any Questions? 16