Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

Recent HPC trial of MIKE 21 Flexible Mesh Series in the UK gives impressive results (DHI).

MIKE by DHI 15th UK User Group Meeting - Tuesday 19 March 2013

Published in: Technology
  • Be the first to comment

  • Be the first to like this


  1. 1. MIKE FM HPC trial at University of SouthamptonLars Steen Sørensen
  2. 2. Basic parallelization concept• The distribution of work is based on the domain decomposition concept (physical sub-domains)• The same program is running on a number of processors, each processor integrates the basic equation in a sub-domain• Data exchange between sub-domains is based on halo layers/elements concept (“ghost”-cells) .• Message passing interface (MPI) standard interface used for communication between processors• I/O is handled on sub-domain level© DHI
  3. 3. Iridis• A 11.760 core HPC cluster with a peak performance of over 105 TFlops• Hosted and operated by the University of Southampton• 924 Intel Westmere compute nodes are available for use by e-Infrastructure South Consortium users with each two 6-core 2.4 GHz Intel Xeon E5645 processors per node• 84 Intel Nehalem compute nodes with two 2,4 GHz 4-core Intel Xeon E5530 processors per node• Approximately 22 GB of usable memory per node. The remaining memory is used to store the . O/S as the nodes are stateless.• Infiniband interconnect, composed of groups of 32 nodes connected by DDR links to a 48 port QDR leaf-switch. The leaf switches then have 4 trunked QDR connections to 4 QDR 48-port core switches (giving 4 redundant pathways for extra bandwidth & resilience).• IBM general parallel file system (GPFS), giving around 110 TB of usable storage divided among the users and split between permanent storage and temporary/scratch space.© DHI
  4. 4. Bristol Channel benchmark descriptionMIKE 21 Spectral Waves FM Element Elements Nodes shapeFull spectral formulation: Mesh A Triangular 10.324 5.71530 frequencies Mesh D Triangular 165.184 84.82216 directions .Time-varying current andwater level includesSimulation period is 24 hourswith mesh A and 6 hours withmesh D© DHI
  5. 5. Bristol Channel benchmark results 60 54 48 42 36 30 I . 24 A C 18 12 6 0 0 6 12 18 24 30 36 42 48 54 60 Number of nodes (12 core each) I: ideal speedup factor, A: small mesh, C: large mesh© DHI
  6. 6. Mediteranean Sea benchmark descriptionMIKE 21 Flow Model FM Element Max. Area Elements Nodes shape Degree2Prediction of tides Mesh A 0.005 Triangular 80.968 41.825 Mesh D Triangular 1.292.116 651.375 0.0003125Simulation period is 2 days .with mesh A and 6 hours withmesh D© DHI
  7. 7. Mediteranean Sea benchmark results 60 54 48 42 36 I 30 . 24 A D 18 12 6 0 0 6 12 18 24 30 36 42 48 54 60 Number of nodes (12 core each) I: ideal speedup factor, A: small mesh, D: large mesh© DHI